Image processing device and associated methodology for generating panoramic images

ABSTRACT

One embodiment of an apparatus includes a reference position receiving unit configured to receive intermediate or end panorama reference position information input by a user, and a control unit configured to control an imaging device to begin generating a plurality of images to be used to generate a panoramic image based on the intermediate or end panorama reference position information input by the user after the reference position receiving unit receives the intermediate or end panorama reference position information.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.16/253,821, filed Jan. 22, 2019, which is a continuation of U.S.application Ser. No. 15/079,578 filed Mar. 24, 2016, which is acontinuation of U.S. application Ser. No. 14/873,548 filed Oct. 2, 2015,which is a continuation of U.S. application Ser. No. 13/551,663 filedJul. 18, 2012 and is based upon and claims the benefit of priority under35 U.S.C. § 119 of Japanese Priority Patent Application JP 2011-168862filed in the Japanese Patent Office on Aug. 2, 2011. The entire contentsof each of which are hereby incorporated by reference.

BACKGROUND

The present technology relates to an image processing device, and moreparticularly relates to an image processing device which handlespanorama images, a control method thereof, and a program encoded on anon-transitory computer readable medium causing a computer to executethe method.

As of recent, imaging apparatuses such as digital still cameras or thelike which generate an image (image data) by imaging a subject such asscenery or the like, and record the generated image as an image file,have come into widespread use. Also, there have been proposed imagingapparatuses which generate multiple images continuously in timesequence, and use the multiple generated images to generate a panoramaimage including the subject over a relative wide range.

For example, there has been proposed an imaging apparatus whichgenerates multiple image while moving the imaging apparatus along an arcof which the axis is behind the imaging apparatus (e.g., the position ofthe photographer), and uses the multiple images to generate a panoramaimage (e.g., Japanese Unexamined Patent Application Publication No.2009-268037).

SUMMARY

According to the above-described related art, a panorama image imagingoperation can be performed by the user moving the imaging apparatusalong an arc while holding the imaging apparatus in the hand, so theuser can perform the imaging operation in a relatively easy manner.

For example, in the event of performing a panorama image imagingoperation with the above related art, the user visually confirms thesubject of the panorama image by looking around, and then performs astart operation for the imaging operation. However, since a panoramaimage is an image including a subject over a relatively wide range, itcan be expected that just the user visually confirming before the startoperation for the imaging operation may not yield the imaging rangewhich the user has intended. Accordingly, in order to generate apanorama image according to user preferences, it is important for apanorama image imaging range according to user preferences to be able tobe easily decided.

It has been found desirable to enable a panorama image imaging rangeaccording to user preferences to be easily decided.

The present invention broadly comprises an image processing device, acontrol method thereof, and a program encoded on a non-transitorycomputer readable medium to cause a computer to execute the controlmethod. In one embodiment, the image processing device includes: anoperation accepting unit configured to accept a deciding operation todecide a reference position in a panorama image after starting animaging operation of the panorama image using a plurality of imagesequentially generated by an imaging unit which consecutively imagessubjects in time sequence, before starting the imaging operation of thepanorama image; and a control unit configured to control deciding of animaging range of the panorama image in the subject, based on the decidedreference position. Accordingly, accepting the deciding operation todecide the reference position in the panorama image after starting thepanorama image imaging operation, before starting of the panorama imageimaging operation, has the effect of deciding the panorama image imagingoperation based on the decided reference position.

In another embodiment, the apparatus includes a reference positionreceiving unit configured to receive intermediate or end panoramareference position information input by a user, and a control unitconfigured to control an imaging device to begin generating a pluralityof images to be used to generate a panoramic image based on theintermediate or end panorama reference position information input by theuser after the reference position receiving unit receives theintermediate or end panorama reference position information.

In a further embodiment, the apparatus includes a reference positionreceiving unit configured to receive panorama reference positioninformation for a plurality of reference points input by a user, and acontrol unit configured to control an imaging device to begin generatinga plurality of images to be used to generate a panoramic image based onthe panorama reference position information for the plurality ofreference points input by the user after the reference positionreceiving unit receives the panorama reference position information forthe plurality of reference points.

In still a further embodiment, the apparatus includes a referenceposition receiving unit configured to receive an end panorama referenceposition image, and a control unit configured to control an imagingdevice to begin generating a plurality of images to be used to generatea panoramic image based on the end panorama reference position imageinput by the user after the reference position receiving unit receivesthe end panorama reference position image, and to control the imagingdevice to end generating the plurality of images when a current imagematches the end panorama reference position image.

The image processing device may further include a display control unitconfigured to display, on an image generated by the imaging unit, anoperation supporting screen representing the decided imaging range,during the imaging operation of the panorama image by the imaging unit.This has the effect of displaying, on an image generated by the imagingunit, an operation supporting screen representing the decided imagingrange, during the imaging operation of the panorama image by the imagingunit.

The display control unit may display, of the subject included in imagesgenerated by the imaging unit, an image to identify a subject includedin the decided imaging range, on the display unit as the operationsupporting screen. This has the effect of displaying, of the subjectincluded in images generated by the imaging unit, an image (operationsupporting image) to identify a subject included in the decided imagingrange.

The image processing device may further include an attitude detectingunit configured to detect change in the attitude of the imagingapparatus, with the display control unit changing the display form ofthe operation supporting screen based on the detected attitude change,which is displayed. This has the effect of changing the display form ofthe operation supporting screen based on the detected attitude change,which is displayed.

The display control unit may display an image representing the outlineof a generally rectangular form corresponding to the panorama image, onthe display unit as the operation supporting screen. This has the effectof displaying an image (operation supporting image) representing theoutline of a generally rectangular form corresponding to the panoramaimage.

The operation accepting unit may accept setting operations to set thelongitudinal direction size of the imaging range of the panorama image,with the control unit deciding the imaging range of the panorama imagebased on the set size and the decide reference position. This has theeffect of deciding the panorama image imaging range based on the setsize and decided reference position.

In a case where the panorama image is generated using a plurality ofimages generated by the imaging unit during turning motion of theimaging apparatus in a particular direction on an axis which is aposition of the imaging apparatus or a position near the imagingapparatus, the operation accepting unit may accept the decidingoperation deciding an end position of the turning movement to be thereference position; with the control unit determining whether or not animage generated by the imaging unit at the time of the decidingoperation being accepted, and an image generated by the imaging unitafter change in attitude of the imaging apparatus after the decidingoperation, match, and in the event that these images match, effectingcontrol to stop the imaging operation of the panorama image by theimaging unit. This has the effect of determining whether an imagegenerated by the imaging unit at the time of the deciding operationdeciding the end position (reference position) of the turning movementof the imaging apparatus being accepted, and an image generated by theimaging unit after change in attitude of the imaging apparatus after thedeciding operation, match, and in the event that these images match, tostop the imaging operation of the panorama image.

In the event that the panorama image is generated using a plurality ofimages generated by the imaging unit during turning motion of theimaging apparatus in a particular direction on an axis which is aposition of the imaging apparatus or a position near the imagingapparatus, the operation accepting unit may accept the decidingoperation deciding an intermediate position of the turning movement tobe the reference position; with the control unit deciding the startposition and end position of imaging operation of the panorama image bythe imaging unit, based on the decided intermediate position. This hasthe effect that, in the event that a deciding operation deciding theintermediate position (reference position) of the turning motion of theimaging apparatus is accepted, the start position and end position ofthe panorama image imaging operation are decided based on the decidedintermediate position.

The image processing device may further include: an attitude detectingunit configured to detect change in the attitude of the imagingapparatus; wherein, in the event that the turning motion of the imagingapparatus in a particular direction on an axis which is a position ofthe imaging apparatus or a position near the imaging apparatus isdetected as the attitude change, the control unit decides the imagingrange of the panorama image such that the particular direction is thelongitudinal direction thereof, and in the event that the turning motionof the imaging apparatus in an orthogonal direction orthogonal to theparticular direction on an axis which is a position of the imagingapparatus or a position near the imaging apparatus is detected as theattitude change, the control unit decides the imaging range of thepanorama image such that the orthogonal direction is the longitudinaldirection thereof. This has the effect that, in the event that turningmotion of the imaging apparatus in a particular direction is detected, apanorama image imaging range with the particular direction as thelongitudinal direction is decided, and in the event that turning motionof the imaging apparatus in an orthogonal direction is detected, apanorama image imaging range with the orthogonal direction as thelongitudinal direction is decided.

The image processing device may further include: a first imaging unit;and a second imaging unit disposed so as to have an optical axisdirection different from the optical axis direction of the imaging unit;wherein, in the event that the panorama image is generated using aplurality of images generated by the imaging unit during turning motionof the imaging apparatus in a particular direction on an axis which is aposition of the imaging apparatus or a position near the imagingapparatus, the operation accepting unit accepts the deciding operationdeciding an intermediate position of the turning movement to be thereference position; and wherein, in the event that the decidingoperation is accepted in a state where the optical axis direction of thesecond imaging unit faces the start position or end position in theimaging operation of the panorama image, the control unit decides theimaging range of the panorama image based on the positional relationbetween the first imaging unit and the second imaging unit. This has aneffect that, in the event that a deciding operation deciding theintermediate position (reference position) of turning motion of theimaging apparatus in a state with the optical axis direction of thesecond imaging unit toward the start position or end position in thepanorama image imaging operation, the panorama image imaging range isdecided based on the placement relation between the first imaging unitand the second imaging unit.

The second imaging unit may generate a plurality of images underexposure conditions different from exposure conditions of the firstimaging unit during imaging operations of the panorama image; with aplurality of images generated by the first imaging unit, and a pluralityof images generated by the second imaging unit, being combined togenerated the panorama image, in which at least a partial region has anexpanded dynamic range. This has the effect that, by combining multipleimages generated by the first imaging unit, and multiple imagesgenerated by the second imaging unit, a panorama image is generated inwhich at least a partial region has an expanded dynamic range.

According to the present technology, an excellent advantage can be hadin that a panorama image imaging range according to user preferences canbe easily decided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a functional configurationexample of an imaging apparatus according to a first embodiment of thepresent technology;

FIG. 2A is a perspective view showing an external view of the imagingapparatus according to the first embodiment of the present technology;

FIG. 2B is another perspective view showing another external view of theimaging apparatus according to the first embodiment of the presenttechnology;

FIG. 3 is a diagram illustrating an example of a settings screen (imagesize setting screen) displayed on an input/output panel of the imagingapparatus according to the first embodiment of the present technology;

FIG. 4A is an image illustrating an example of a relation between animaging operation in the case of generating a panorama image using theimaging apparatus according to the first embodiment of the presenttechnology, and the panorama image taken by this imaging operation;

FIG. 4B is another image illustrating an example of a relation betweenan imaging operation in the case of generating a panorama image usingthe imaging apparatus according to the first embodiment of the presenttechnology, and the panorama image taken by this imaging operation;

FIG. 5A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the first embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 5B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 6A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the first embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 6B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 7A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the first embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 7B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 8A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the first embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 8B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 9A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the first embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 9B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 10A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the first embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 10B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 11 is a flowchart illustrating an example of processing proceduresof imaging control processing by the imaging apparatus according to thefirst embodiment of the present technology;

FIG. 12 is a flowchart illustrating an example of processing proceduresof imaging control processing by the imaging apparatus according to thefirst embodiment of the present technology;

FIG. 13 is a block diagram illustrating a functional configurationexample of an imaging apparatus according to a second embodiment of thepresent technology;

FIG. 14A is a diagram schematically illustrating an example of imageholding in a case of holding an image in a matching image detecting unitaccording to the second embodiment of the present technology;

FIG. 14B is another diagram schematically illustrating an example ofimage holding in a case of holding an image in a matching imagedetecting unit according to the second embodiment of the presenttechnology;

FIG. 15A is a diagram schematically illustrating matching processingwith the matching image detecting unit according to the secondembodiment of the present technology;

FIG. 15B is another diagram schematically illustrating matchingprocessing with the matching image detecting unit according to thesecond embodiment of the present technology;

FIG. 16 is a flowchart illustrating an example of processing proceduresof imaging control processing by the imaging apparatus according to thesecond embodiment of the present technology;

FIG. 17 is a flowchart illustrating an example of processing proceduresof imaging control processing by the imaging apparatus according to thesecond embodiment of the present technology;

FIG. 18A is an image illustrating an example of a relation between theattitude of the imaging apparatus according to a third embodiment of thepresent technology, and the display screen displayed on the input/outputpanel;

FIG. 18B is another image illustrating an example of a relation betweenthe attitude of the imaging apparatus according to a third embodiment ofthe present technology, and the display screen displayed on theinput/output panel;

FIG. 19A is an external configuration diagram showing an external viewof the imaging apparatus according to the fourth embodiment of thepresent technology;

FIG. 19B is another external configuration diagram showing anotherexternal view of the imaging apparatus according to the fourthembodiment of the present technology;

FIG. 20 is a block diagram illustrating a functional configurationexample of the imaging apparatus according to the fourth embodiment ofthe present technology;

FIG. 21 is a diagram illustrating an example of a settings screen(shooting method setting screen) displayed on the input/output panel ofthe imaging apparatus according to the fourth embodiment of the presenttechnology;

FIG. 22A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the fourth embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 22B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the fourthembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 23A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the fourth embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 23B is another diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the fourthembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 24A is a diagram schematically illustrating a transition example ofthe attitude of the imaging apparatus according to the fourth embodimentof the present technology and a display screen displayed on theinput/output panel;

FIG. 24B is another a diagram schematically illustrating a transitionexample of the attitude of the imaging apparatus according to the fourthembodiment of the present technology and a display screen displayed onthe input/output panel;

FIG. 25A is a diagram illustrating an example of combining a panoramaimage using images generated by multiple imaging units according to thefourth embodiment of the present technology;

FIG. 25B is another diagram illustrating an example of combining apanorama image using images generated by multiple imaging unitsaccording to the fourth embodiment of the present technology;

FIG. 25C is a further diagram illustrating an example of combining apanorama image using images generated by multiple imaging unitsaccording to the fourth embodiment of the present technology;

FIG. 26A is external configuration diagram showing an external view ofan imaging apparatus according to the fourth embodiment of the presenttechnology;

FIG. 26B is another external configuration diagram showing anotherexternal view of an imaging apparatus according to the fourth embodimentof the present technology;

FIG. 27A is a diagram illustrating a display example of an operationassisting screen displayed on the imaging apparatus according to amodification of an embodiment of the present technology;

FIG. 27B is another diagram illustrating a display example of anoperation assisting screen displayed on the imaging apparatus accordingto a modification of an embodiment of the present technology;

FIG. 28A is a diagram illustrating a display example of an operationassisting screen displayed on the imaging apparatus according to amodification of an embodiment of the present technology;

FIG. 28B is another diagram illustrating a display example of anoperation assisting screen displayed on the imaging apparatus accordingto a modification of an embodiment of the present technology;

FIG. 29A is a diagram screen displayed on the imaging apparatusaccording to a modification of an embodiment of the present technology;

FIG. 29B is another diagram screen displayed on the imaging apparatusaccording to a modification of an embodiment of the present technology;

FIG. 30A is a diagram illustrating a display example of an operationassisting screen displayed on the imaging apparatus according to amodification of an embodiment of the present technology;

FIG. 30B is another diagram illustrating a display example of anoperation assisting screen displayed on the imaging apparatus accordingto a modification of an embodiment of the present technology;

FIG. 31 is a diagram illustrating a display example of an operationassisting screen displayed on the imaging apparatus according to amodification of an embodiment of the present technology;

FIG. 32A is a diagram illustrating the relation between the imagingrange of a panorama image generated by an imaging apparatus according toa modification of an embodiment of the present technology, and animaging operation;

FIG. 32B is another diagram illustrating the relation between theimaging range of a panorama image generated by an imaging apparatusaccording to a modification of an embodiment of the present technology,and an imaging operation;

FIG. 33A is a diagram schematically illustrating an imaging operationperformed at the time of generating a panorama image by an imagingapparatus according to a modification of an embodiment of the presenttechnology; and

FIG. 33B is another diagram schematically illustrating an imagingoperation performed at the time of generating a panorama image by animaging apparatus according to a modification of an embodiment of thepresent technology.

DETAILED DESCRIPTION OF EMBODIMENTS

The following is a description of embodiments for carrying out thepresent technology (hereinafter referred to as “embodiments”).Description will be made in the following order.

-   1. First Embodiment (example of deciding imaging range of panorama    image based on a reference position in the panorama image decided by    user operations)-   2. Second Embodiment (example of deciding ending timing of imaging    operation of panorama image based on matching processing using a    reference position in the panorama image decided by user operations)-   3. Third Embodiment (example of deciding intermediate position in    panorama image as reference position)-   4. Fourth Embodiment (example of performing imaging operation of    panorama image using imaging apparatus having multiple imaging    units)-   5. Modifications

1. First Embodiment Functional Configuration Example of ImagingApparatus

FIG. 1 is a block diagram illustrating a functional configurationexample of an imaging apparatus 100 according to a first embodiment ofthe present technology.

The imaging apparatus 100 includes a lens unit 111, an imaging device112, an image processing unit 113, an image combining unit 120, arecording control unit 130, a recording medium 140, an attitudedetecting unit 150, a control unit 160, a display control unit 170, adisplay unit 180, and an operation accepting unit 190. The imagingapparatus 100 can be realized by a digital still camera which is capableof generating multiple images (image data) by imaging a subject, forexample, and performing various types of image processing regardingthese multiple images.

The lens unit 111 is configured of multiple lenses which collect lightfrom the subject (zoom lens, focusing lens, etc.), and supplies thelight from the subject that has been input via these lenses and an iristo the imaging device 112.

Based on the control of the control unit 160, the imaging device 112converts the incident light form the subject to generate an image (imagedata), and the generated image is supplied to the image processing unit113. Specifically, an optical image of the subject input via the lensunit 111 is imaged on the imaging face of the imaging device 112, and animage (image data) is generated by the imaging device 112 performing animaging operation in this state. Also, in the event that a panoramaimage imaging mode has been set, the imaging device 112 sequentiallygenerates multiple images by consecutively imaging the subject in timesequence, and supplies the multiple imaged images to the imageprocessing unit 113. Examples of the imaging device 112 which can beused include a CCD (Charge Coupled Device), a CMOS (Complementary MetalOxide Semiconductor), or the like.

Based on the control of the control unit 160, the image processing unit113 performs various types of image processing based on the image (imagedata) generated by the imaging device 112, and supplies the image (imagedata) which has been subjected to the image processing to the imagecombining unit 120 and display control unit 170.

The image processing unit 113 temporarily holds the image (image data)generated by the imaging device 112 in image memory (not shown) forexample, and performs image processing on the image held therein. Theimage memory is configured of DRAM (Dynamic Random Access Memory), forexample.

For example, the image processing unit 113 performs image processing fordisplay regarding the image (image data) generated by the imaging device112, and outputs the image subjected to this image processing to thedisplay control unit 170 so as to be displayed on the display unit 180.Also, in the event that the panorama image imaging mode has been set,the image processing unit 113 perform image processing for panoramaimage regarding the image (image data) generated by the imaging device112, and outputs to the image combining unit 120.

Also, in the event that the panorama image imaging mode has been set,the image processing unit 113 detects the amount of movement and thedirection of movement between images adjacent on the temporal axis, forthe images held in the image memory (not shown). The image processingunit 113 then outputs information relating to the amount of movement andthe direction of movement (movement information) that has been detectedto the image combining unit 120 and control unit 160. For example, theimage processing unit 113 performs matching processing between thepixels making up the two adjacent images (i.e., matching processing todistinguish imaging regions of the same subject), and calculates thenumber of pixels for the movement between the images. In the matchingprocessing, processing is performed assuming that the subject isbasically still. Note that in the event that the subject includes movingbodies, motion vectors which differ from the motion vector of theoverall image are detected, and the motion vectors corresponding tothese moving bodies are processed as being exempt from detection. Thatis to say, only a motion vector (global motion vector, also abbreviatedto “GMV”) corresponding to the motion of the overall image occurring dueto motion of the imaging apparatus 100 is detected.

Based on the control of the control unit 160, the image combining unit120 generates a panorama image using the multiple images supplied fromthe image processing unit 113, and supplies the generated panorama imageto the recording control unit 130. For example, the image combining unit120 temporarily holds the multiple images supplied from the imageprocessing unit 113 in image memory (not shown), and generates apanorama image using these multiple images that are held.

In another embodiment, the apparatus 100 is in communication with aserver, and transmits the multiple images supplied from the imageprocessing unit 113 to the server. The server then generates thepanoramic image and sends the panoramic image to apparatus 100. Theapparatus 100 may be a mobile phone in such an embodiment.

Also, based on the analysis results output from the control unit 160(the analysis results of amount in change of the attitude of the imagingapparatus 100), the image combining unit 120 calculates the regions tobe combined in each of the multiple images held in the image memory (notshown). The image combining unit 120 then extracts images from theregions to be combined of each of the multiple images, and generates apanorama image by combining these extracted images. In this case, theimage combining unit 120 overlays and combines the extracted imagesbased on the movement information (amount of movement and the directionof movement) output from the image processing unit 113.

Under control of the control unit 160, the recording control unit 130performs control to record the image (image data) subjected to imageprocessing by the image processing unit 113 in the recording medium 140.Also, in the event that the panorama image imaging mode has been set,the recording control unit 130 records the panorama image (image data)generated by the image combining unit 120 in the recording medium 140.

The recording medium 140 is a device which records images subjected toimage processing by the image processing unit 113 and panorama imagesgenerated by the image combining unit 120 as image files (imagecontents), under control of the recording control unit 130. For example,various types of data such as JPEG format image data or the like arerecorded in the recording medium 140. The recording medium 140 may bebuilt into the imaging apparatus 100, or may be detachable from theimaging apparatus 100. Examples of the recording medium 140 which can beused include various types of recording media such as semiconductormemory, optical recording media, magnetic disks, HDD (Hard Disk Drive),and so forth. Note that examples of optical recording media includerecordable DVDs (Digital Versatile Disk), recordable CDs (Compact Disc),Blu-ray discs (registered trademark), and so forth.

The attitude detecting unit 150 detects change in the imaging apparatus100 by detecting acceleration, motion, tile, and the like of the imagingapparatus 100, and outputs attitude information relating to change inattitude that has been detected to the control unit 160. An example ofthe attitude detecting unit 150 which can be used is a gyro sensor. Thisgyro sensor detects the angular acceleration of the imaging apparatus100, whereby change in attitude of the imaging apparatus 100 isdetected. Note that an arrangement may be made where a sensor other thana gyro sensor (e.g., an acceleration sensor) is used to detect theacceleration, motion, tilt, and so forth of the imaging apparatus 100,so as to detect the attitude and the change thereof of the imagingapparatus 100.

Under control of the control unit 160, display control unit 170 displaysvarious types of images on the display unit 180. For example, thedisplay control unit 170 displays the image supplied from the imageprocessing unit 113 as a display image (e.g., a live view image (alsocalled “LV image”)) on the display unit 180. The display control unit170 displays various setting screens (e.g., an image size setting screen300 shown in FIG. 3) on the display unit 180. Also, the display controlunit 170 combines images (e.g., operation supporting images 317 and 318shown in FIG. 6B) with images supplied from the image processing unit113 (e.g., LV images) so as to be displayed on the display unit 180.

For example, while performing a panorama image imaging operation, thedisplay control unit 170 overlaps an operation supporting imagerepresenting the imaging range of the panorama image upon an imagesupplied from the image processing unit 113 (e.g., LV image). Thisoperation supporting image is an image for identifying, out of thesubjects included in an LV image, an image of a subject included in theimaging range of the panorama image (e.g., an image representing agenerally rectangular shape corresponding to the panorama image) asshown in FIGS. 6B, 7B, 8B, 9B, and 10B. Also, the display control unit170 changes the display form of the operation supporting image based onchange in the attitude detected by the attitude detecting unit 150, asshown in FIGS. 6B, 7B, 8B, 9B, and 10B. That is to say, the displaycontrol unit 170 causes display to be performed such that the operationsupporting image moves based on the analysis results output from thecontrol unit 160 (the analysis results of the amount of change inattitude of the imaging apparatus 100).

The display unit 180 is a display unit which displays various types ofimages under control of the display control unit 170. The display unit180 sequentially displays images generated by the imaging device 112 asLV images. Note that an example of the display unit 180 which can beused is a display panel such as an organic EL (Electro Luminescence)panel or the like. Also, a touch panel whereby the user can performoperations by bringing his/her fingers into contact or in closeproximity to the display screen thereof may be used, as shown in FIG. 3.

The operation accepting unit 190 is an operation accepting unit whichaccepts operation input from user operations, and outputs operationsignals corresponding to the accepted operation input to the controlunit 160. The operation accepting unit 190 includes operating memberssuch as a shutter button 191 and a dial 192 and so forth. As shown inFIG. 6A, for example, the operation accepting unit 190 acceptsoperations for deciding a reference position (e.g., an end position) ina panorama image after panorama image imaging operations have started,before starting the panorama image imaging operation. Also, theoperation accepting unit 190 accepts setting operations to set the size(image size) of the panorama image in the longitudinal direction of theimaging range of the panorama image, as shown in FIG. 3 for example.Note that the display unit 180 and operation accepting unit 190 may beintegrally formed as an input/output panel 200, as shown in FIG. 2B.

The control unit 160 controls the members of the imaging apparatus 100based on the operating signals from the operation accepting unit 190 andthe attitude information from the attitude detecting unit 150. Forexample, in the event that an imaging mode setting operation has beenaccepted by the operation accepting unit 190, the control unit 160 setsthe imaging mode corresponding to that setting operation. Note that withthe first embodiment of the present technology, a case will be describedin which the panorama image imaging mode is set to generate a panoramaimage, as an example. Also, in the event that a setting operation forimage size (shown in FIG. 3) is accepted by the operation accepting unit190, the image size is set according to that setting operation.

Also, based on the attitude information output from the attitudedetecting unit 150 the control unit 160 analyzes the amount of change ofattitude of the imaging apparatus 100 (direction of movement, amount ofmovement, etc.), and outputs the analysis results thereof to the imagecombining unit 120 and the display control unit 170.

Also, in the event that a deciding operation to decide a referenceposition (e.g., end position) has been accepted by the operationaccepting unit 190, the control unit 160 effects control to decide theimaging range of the panorama image based on the decided referenceposition. As shown in FIG. 6A, the imaging range of the panorama image(e.g., a rotational angle identified by a start position 423 and endposition 422) is decided based on a set image size (e.g., θ1) and thedecided reference position (e.g., end position 422). This having beendecided decides the panorama image imaging range of the subject.

External Configuration Example of Imaging Apparatus

FIGS. 2A and 2B are perspective views showing external views of theimaging apparatus 100 according to the first embodiment of the presenttechnology. FIG. 2A is a perspective view illustrating the outer view ofthe frontal side of the imaging apparatus 100 (i.e., the face where thelens unit 111 to be directed toward the subject is provided). Also, FIG.2B is a perspective view illustrating the outer view of the rear side ofthe imaging apparatus 100 (i.e., the face where the input/output panel200 to be directed toward the photographer is provided).

The imaging apparatus 100 has a lens unit 111, shutter button 191, dial192, and input/output panel 200. Note that while the imaging apparatus100 has other operating members such as a power switch, mode switchoverswitch, zoom button, and so forth, illustration and description thereofwill be omitted here.

The shutter button 191 is a button pressed by the user at the time ofrecording an image (image data) generated by the imaging device 112 asan image content. For example, in a case where a still image imagingmode has been set to record a still image, and the shutter button 191 ishalf-pressed, focusing control is effected to perform autofocusing.Also, in the event that the shutter button 191 is fully pressed, thefocus control is effected, and the image generated by the imaging device112 at the time of fully pressing is recorded in the recording medium140 as an image file (still image file). Pressing operations of theshutter button 191 in the event that the panorama image imaging mode hasbeen set will be described in detail with reference to FIGS. 5A through10B and others.

The dial 192 is a dial used to perform various types of adjustment orthe like. For example, the dial 192 is operated to set the imaging range(image size) of a panorama image.

The input/output panel 200 displays various types of images, and alsoaccepts operation input from the user by detecting touch operations onthe input/output panel 200. The input/output panel 200 is realized by atouch panel, for example. The input/output panel 200 corresponds to thedisplay unit 180 and operation accepting unit 190 shown in FIG. 1.

Note that with the description of the first embodiment of the presenttechnology, a rotating operation where the imaging apparatus 100 isrotated in a particular direction (e.g., horizontal direction, verticaldirection) with the current position of the imaging apparatus 100 (or aposition nearby the imaging apparatus 100 (e.g., a position behind)) asan axis will be called a “swing operation”. Also, the operationdirection thereof will be called “swing direction”. Such rotation ofapparatus 100 may be performed by the user, by another apparatusattached to the apparatus 100, or by a rotary actuator included inapparatus 100.

Example of Image Size Setting Screen

FIG. 3 is a diagram illustrating an example of a setting screen (imagesize setting screen 300) displayed on the input/output panel 200 of thefirst embodiment of the present technology.

The image size setting screen 300 is a screen displayed on theinput/output panel 200 at the time of setting the imaging range (imagesize) of a panorama image. For example, the image size setting screen300 is displayed immediately following performing a setting operationfor the panorama image imaging mode. Also, the imaging range (imagesize) of the panorama image set at the image size setting screen 300 isthe imaging range (image size) in the swing direction of the imagingapparatus 100.

The image size setting screen 300 is provided with an image sizespecifying bar 301, a specified position identifier 302, an OK button303, and a return button 304.

The image size specifying bar 301 is a bar used to specify the panoramaimage imaging range (image size), and is displayed with the specifiedposition identifier 302 displayed thereupon. For example, in the eventthat up to 360° can be specified as the panorama image imaging range(image size), “0° ” is displayed at one end of the image size specifyingbar 301 and “360° ” at the other end thereof. Also, numerical values(e.g., 60°, 120°, and so on) are displayed at fixed intervals (e.g., at60° intervals) are displayed on the image size specifying bar 301.

The panorama image imaging range (image size) can be specified by theuser moving the specified position identifier 302 along the image sizespecifying bar 301 to the desired position. This moving operation of thespecified position identifier 302 can be performed by operating the dial192, or a touch operation on the input/output panel 200, for example.

The OK button 303 is a button pressed after a specification operationhas been made to specify the panorama image imaging range (image size)to OK what has been specified. Also, information (image sizeinformation) relating to the panorama image imaging range (image size)decided by the operation of pressing the OK button 303 is output to thecontrol unit 160 and held.

The return button 304 button is a button pressed in the case ofreturning to the display screen displayed immediately before, forexample.

Note that with this example, an example is illustrated of specifying theimage size using the image size specifying bar 301, but the image sizemay be specified by button pressing operations, or input operations inwhich numerical values are input, for example. For example, a selectingoperation may be made in which the user selects a desired image sizefrom multiple types of image sizes (e.g., image size button pressingoperation). Also, the image size may be set by the user performing aninput operation (e.g., input operation of numerical values of the imagesize (angle)).

Also, with this example, an example is illustrated of specifying thepanorama image imaging range (image size) by user operations, but theimaging range may be set beforehand. For example, 180° may be set as areference size.

Example of Relation Between Imaging Operation and Panorama Image

Next, a case of performing an imaging operation using the imagingapparatus 100 will be described. FIGS. 4A and 4B are images illustratingan example of a relation between an imaging operation in the case ofgenerating a panorama image using the imaging apparatus 100 according tothe first embodiment of the present technology, and the panorama imagetaken by this imaging operation.

FIG. 4A is a plan view illustrating a place 400 which is an object ofimaging in the case of generating a panorama image using the imagingapparatus 100. The place 400 is a place including mountains, buildings401 and 402, a large tree 403, a lake, and so forth, which areillustrated in a simplified manner in FIG. 4A to facilitate description.Also, FIG. 4A illustrates an example of performing a swing operation ofthe imaging apparatus 100 from the left side to the right side in a casewhere 180° has been set as the image size. In FIG. 4A, an example isillustrated of generating a panorama image by the user operating theimaging apparatus 100 such that the buildings 401 and 402 and the largetree 403 will be included in the panorama image. Note that the termsleft and right as used in the embodiments of the present technology meanleft and right with the user operating the imaging apparatus 100 as areference.

FIG. 4B illustrates a panorama image 410 generated by the imagingoperation shown in FIG. 4A. That is to say, the panorama image 410 is apanorama image including the buildings 401 and 402 and the large tree403. We will thus assume a case where a user 420 wandering along thelakeside desires a panorama image including the buildings 401 and 402and the large tree 403. In this case, the user 420 has to start thepanorama image imaging operation at a start position 405 of the imagingoperation and end the panorama image imaging operation at an endposition 406 of the imaging operation.

In order to perform such an imaging operation, the user 420 has to lookaround the place 400, comprehend the imaging range of the panoramaimage, and start the imaging operation. However, it can be expected tobe difficult to look around the place 400 and accurately comprehend theimaging range of the panorama image visually. Also, in the event thataccurately comprehending the imaging range of the panorama image isdifficult, the panorama image which the user 420 desires (a panoramaimage including the buildings 401 and 402 and the large tree 403) maynot be generated.

Now, with the first embodiment of the present technology, an examplewill be illustrated where the deciding operation to decide the referenceposition in the panorama image after the panorama image imagingoperation has started, is accepted before starting the panorama imageimaging operation, and the panorama image imaging range is decided basedon the decided reference position.

Example of Panorama Image Imaging Operation

FIGS. 5A through 10B are diagrams illustrating a transition example ofthe attitude of the imaging apparatus 100 according to the firstembodiment of the present technology and a display screen displayed onthe input/output panel 200. That is to say, FIGS. 5A, 6A, 7A, 8A, 9A,and 10A illustrate an example of transition of the attitude of theimaging apparatus 100. Note that in FIGS. 5A, 6A, 7A, 8A, 9A, and 10A,only the buildings 401 and 402 and the large tree 403 in the place 400shown in FIG. 4A are shown, and others are omitted from illustration.Also, FIGS. 5B, 6B, 7B, 8B, 9B, and 10B illustrate an example oftransition of the display screen displayed on the input/output panel 200in accordance with change in the attitude of the imaging apparatus 100.FIGS. 5A through 10B illustrate an example of an imaging operation in acase that the image size has been set to 180°.

FIG. 5A illustrates a state of the imaging apparatus 100, and the user420 in a state of holding the imaging apparatus 100 in both hands, asviewed from above in a simplified manner. We will say that the user 420is watching the LV image displayed on the input/output panel 200 whileconfirming the end position of the panorama image, for example.

FIG. 5B illustrates a display screen 310 displayed before the decidingoperation to decide the end position of the panorama image is performed.The display screen 310 has the subject which is currently to be imageddisplayed as an LV image, and a message display region 311 displayedsuperimposed on the LV image.

The message display region 311 is a region where messages for supportingoperations performed by the user 420 are displayed. A message promptingthe user to decide the end position of the panorama image imagingoperation, for example, is displayed on the message display region 311.

For example, we will assume a case where the user 420 has set thepanorama image imaging mode. For example, upon turning the power of theimaging apparatus 100 on, the imaging apparatus 100 starts up, and an LVimage is displayed on the input/output panel 200. The 420 decides theend position (the position corresponding to the right end of thepanorama image) out of the imaging range corresponding to the panoramaimage, while viewing the LV image displayed on the input/output panel200. For example, in the case of deciding an imaging range including thelarge tree 403 as the end position as shown in FIG. 5B, the user 420performs a full pressing operation (deciding operation) of the shutterbutton 191 in the state shown in FIG. 5A. In the event that a decidingoperation of the end position has thus been performed, the control unit160 decides the attitude of the imaging apparatus 100 at the time thatthis deciding operation was made to be the reference attitude of theimaging apparatus 100. That is to say, the attitude relating to theattitude information output from the attitude detecting unit 150 at thetime that this deciding operation was performed is decided as thereference attitude of the imaging apparatus 100. Note that an example ofa display screen displayed on the input/output panel 200 after the user420 decides the end position of the panorama image imaging operation isillustrated in FIG. 6B.

FIG. 6A illustrates a state of the user 420 having full-pressed theshutter button 191 in the state shown in FIG. 5A, in a simplifiedmanner. In this example, we will say that the set image size is θ1(i.e., 180°), and description will be made regarding a method to decidethe start position of the imaging operation in the horizontal direction.

For example, let us assume a case where the user 420 has full-pressedthe shutter button 191 in the state shown in FIG. 5A. In this case, thestart position 423 of the imaging operation is decided based on the setimage size θ1, with the optical axis direction from the position 421 atthe time of that operation (direction of end position 422) as areference. Specifically, as shown in FIG. 6A, a position reached byrotating by an amount of θ1 in the swing direction on the rotationalaxis of the position 421 at the time of having performed thefull-pressing operation of the shutter button 191, is decided as thestart position 423 of the imaging operation. Note that in the case thatthe position 421 (the position of the 420 (i.e., the position of theimaging apparatus 100)) at the time of having performed thefull-pressing operation of the shutter button 191 serves as therotational axis, the position in the optical axis direction thereof isdecided as the end position 422 of the imaging operation. The startposition of the imaging operation in the vertical direction can bedecided in the same way.

FIG. 6B shows an operation assisting screen 315 displayed in the eventthat a deciding operation to decide the end position of the panoramaimage imaging operation has been performed. The operation assistingscreen 315 displays the subject which is currently to be imageddisplayed as an LV image, and a message display region 316 and operationsupporting images 317 and 318 superimposed on the LV image.

The message display region 316 displays a message to the effect that theend position of the panorama image imaging operation has been decided,and a message prompting the user to move the imaging apparatus 100 tothe start position of the panorama image imaging operation.

The operation supporting images 317 and 318 are images for supportinguser operations relating to the panorama image imaging operation (swingoperation of the imaging apparatus 100). For example, images foridentifying subjects included in the panorama image imaging range(dotted lines representing a generally rectangular outline correspondingto the panorama image) are displayed as the operation supporting images317 and 318.

Now, at the time of deciding the end position of the panorama imageimaging operation, the swing direction (horizontal direction or verticaldirection) of the imaging apparatus 100 is undecided, so the operationsupporting images 317 and 318 alone are displayed without displayingoperation supporting images corresponding to either swing direction.Also, an operation supporting image is displayed corresponding to one ofthe swing directions in accordance with change in the attitude of theimaging apparatus 100.

That is to say, in the event that the swing operation of the imagingapparatus 100 in the horizontal direction has been detected, a panoramaimage imaging range with the horizontal direction as the longitudinaldirection is decided, and an operation supporting image corresponding tothe horizontal direction is displayed. Also, in the event that the swingoperation of the imaging apparatus 100 in the vertical direction hasbeen detected, a panorama image imaging range with the verticaldirection as the longitudinal direction is decided, and an operationsupporting image corresponding to the vertical direction is displayed.

Note that a display method of an operation supporting imagecorresponding to change in the attitude of the imaging apparatus 100will be described in detail with reference to FIG. 7A through FIG. 10B.Also, in the event that the swing direction of the imaging apparatus 100(horizontal direction of vertical direction) has been set beforehand, anarrangement may be made where an operation supporting image is displayedcorresponding to the set swing direction at the time of deciding the endposition of the panorama image imaging operation.

Thus, after full-pressing of the shutter button 191, the operationassisting screen 315 is displayed on the input/output panel 200, so theuser 420 can easily perform the swing operation of the imaging apparatus100 following the operation assisting screen 315. Also, an example of adisplay screen displayed on the input/output panel 200 after the userhas started the swing operation will be illustrated in FIG. 7B andothers.

FIG. 7A shows a state in which the user 420 is moving the imagingapparatus 100 in the direction of the arrow 424 from the state shown inFIG. 6A, in a simplified manner.

FIG. 7B shows an operation assisting screen 320 displayed on theinput/output panel 200 immediately after the user 420 has moved theimaging apparatus 100 in the direction of the arrow 424. Note thatmotion of the imaging apparatus 100 after the full-pressing operation ofthe shutter button 191 is determined by the control unit 160 based onthe attitude information from the attitude detecting unit 150.

The operation assisting screen 320 displays the subject which iscurrently to be imaged displayed as an LV image, and a message displayregion 321 and operation supporting images 322 and 324 superimposed onthe LV image. The message display region 321 displays a message to theeffect of displaying the panorama image imaging range, and a messageprompting the user to move the imaging apparatus 100 to the startposition of the panorama image imaging operation.

The operation supporting images 322 through 324 are images forsupporting user operations relating to the panorama image imagingoperation (swing operation of the imaging apparatus 100), as describedabove. Specifically, the operation supporting images 322 and 323 areimages for identifying the subject included in the panorama imageimaging range, and may be dotted lines representing the outline of agenerally rectangular shape corresponding to the panorama image (overallrectangular-shaped dotted lines). Also, the operation supporting images322 and 323 are positioned on the LV image displayed on the operationassisting screen 320. Also, the operation supporting image 324 is asolid line indicating the center position in the panorama image (centerposition in the swing direction), and is situated on the LV imagedisplayed on the operation assisting screen 320. Thus, the user 420 caneasily comprehend the intermediate position in the panorama image by theoperation supporting image 324 being displayed.

Now, the display position of the operation supporting image 324 isdetermined based on the set image size θ1 (i.e.,) 180°), and the endposition 422 of the imaging operation. That is to say, the position“θ1/2” from the end position 422 of the imaging operation is decided asthe display position for the operation supporting image 324.

For example, as shown in FIG. 7A, a vertical direction range and centerposition are schematically near the middle of the panorama image imagingrange, as the imaging range thereof. Also, as shown in FIG. 8B, avertical direction range and end position (left end portion) areschematically near the end position (left end portion) of the panoramaimage imaging range, as the imaging range thereof.

Now, a display method of the operation support images (operation supportimages 322, 323, etc.) will be described. An operation support image isdisplayed as a reference for the current operating position as to theoverall amount of operation which has to be performed in the panoramaimage imaging operation (e.g., rotational angle of the swing operation).For example, the end position of the panorama image imaging operation isset as a reference position, and based on the amount of operation fromthis reference position (end position), an operation support image forone of the horizontal direction and vertical direction is decided as anoperation support image to be displayed.

Specifically, the current amount of operation is calculated by thecontrol unit 160, based on the detection results of amount of movementand direction of movement (detection results of the image processingunit 113) between images adjacent on the temporal axis (images generatedby the imaging device 112). The control unit 160 then decides theoperation support image (horizontal direction or vertical direction) tobe displayed, based on the current operation amount, and changes thedisplay state of the operation support image. For example, a motionvector corresponding to the overall motion of the image occurring due tothe imaging apparatus 100 moving (i.e., a GMV) is detected as the amountof movement and direction of movement thereof. Also, the control unit160 may calculate the current amount of operation based on the angularvelocity detected by the attitude detecting unit 150. Further, thecontrol unit 160 may calculate the current amount of operation using theamount of movement and direction of movement thereof and the angularvelocity detected by the attitude detecting unit 150. In this case, theuser can easily comprehend the panorama image imaging range bydisplaying an operation support image.

Also, the operation support image may be displayed in a different mannerdepending on whether an operation support image from the operation ofdeciding the end position 422 up throughout the swing operation to thestart position 423, or an operation support image from the startposition 423 throughout the swing operation to the end position 422. Forexample, the operation support image may be made to be displayed in ablinking manner for the operation support image from the operation ofdeciding the end position 422 up throughout the swing operation to thestart position 423. Thus, by displaying the operation support imagewhile performing the swing operation for preparation, the user can benotified that this is not a panorama image imaging operation butmovement as preparation thereof.

Also, in the event that the imaging apparatus 100 has reached the startposition 423 and a starting instruction operation has been performed forthe panorama image imaging operation by the user 420, the display formof the operation support image may be changed from the blinking displayto normal display (constant on). That is to say, the operation supportimage may be displayed normally during the swing operation from thestart position 423 to the end position 422 (panorama image imagingoperation). Accordingly, by changing the display form of the operationsupport image, whether currently preparing for the panorama imageimaging operation or whether currently performing the panorama imageimaging operation can be readily comprehended by the user.

FIG. 8A illustrates a state of the user 420 moving the imaging apparatus100 to the start position 423 of the imaging operation. FIG. 8B shows anoperation assisting screen 325 displayed on the input/output panel 200at the time of the user 420 moving the imaging apparatus 100 to thestart position 423 of the imaging operation. Note that whether or notthe imaging apparatus 100 has reached the start position 423 isdetermined by the control unit 160 based on the attitude informationfrom the attitude detecting unit 150.

An operation assisting screen 325 has operation supporting images 327through 329 displayed instead of the operation supporting images 322 and323 in the operation assisting screen 320 in FIG. 7B. The operationsupporting images 327 and 329 are operation supporting images indicatingvertical direction imaging range in the panorama image, and theoperation supporting image 328 is an operation supporting imageindicating the imaging range at the left edge of the panorama image.

A message display region 326 displays a message to the effect ofprompting a start position deciding operation for the panorama imageimaging operation based on the operation supporting images 327 through329, and a message prompting the user to start the panorama imageimaging operation after the deciding operation.

Thus, in the event of the imaging apparatus 100 having been moved to thestart position 423 of the imaging operation, a message prompting thestart position deciding portion for the panorama image imaging operationis displayed. Also, along with this display, a message to the effect ofswinging the imaging apparatus 100 in the opposite direction as theswinging direction so far to start the imaging operation is displayed.

Thus, in the event that the user 420 performs a full-press operation ofthe shutter button 191 after the imaging apparatus 100 is moved to thestart position 423 of the imaging operation and the operation assistingscreen 325 is displayed on the input/output panel 200, the panoramaimage imaging operation is started. Also, upon the panorama imageimaging operation being started, the operation supporting images 327through 329 go from a blinking display to a constantly on state. Also,the operation supporting images 327 through 329 move in accordance withthe motion of the imaging apparatus 100.

Now, in the event that the user 420 has performed a start positiondeciding operation (full-press operation of the shutter button 191) forexample, it can be assumed that the position at the time of deciding maybe different from the start position 423. For example, it can be assumedthat the position may be shifted several degrees (e.g., one to fivedegrees) from the start position 423, with the position 421 as thecenter. In this case, the set image size may be changed based on thestart position deciding operation by the user 420. For example, let usassume a case where the position at the time of the start positiondeciding operation is shifted +5° (center angle centered on the position421) from the start position 423. In this case, the set image size(180°) can be changed to 185°.

Note that with this example, an example is illustrated where thefull-press operation of the shutter button 191 by the user 420 is thestart position deciding operation for the panorama image imagingoperation, but an arrangement may be made wherein the imaging apparatus100 automatically decides the start position at the time of havingreached the start position 423. In this case, only a message to theeffect that the panorama image imaging operation will be started isdisplayed, and the panorama image imaging operation is startedautomatically. Thus, upon the panorama image imaging operation beingstarted, the operation supporting images change in accordance with themovement of the imaging apparatus 100. An example thereof will bedescribed with reference to FIGS. 9A and 9B.

FIG. 9A illustrates the transition of the imaging apparatus 100 movingfrom the imaging operation start position 423 to the imaging operationend position 422. FIG. 9B shows an operation assisting screen 330displayed during the movement of the imaging apparatus 100 from theimaging operation start position 423 to the imaging operation endposition 422. In this way, after then imaging apparatus 100 has startedthe imaging operation from the imaging operation start position 423, theoperation assisting screen 330 is displayed in accordance with themovement of the imaging apparatus 100 until the imaging apparatus 100reaches the imaging operation end position 422.

The operation assisting screen 330 displays the subject which iscurrently to be imaged displayed as an LV image, and a message displayregion 331, operation supporting images 332 and 333, and an arrow 334superimposed on the LV image. The message display region 331 displays amessage to the effect of performing a swing operation of the imagingapparatus 100 to the panorama image imaging operation end position.

The arrow 334 is an arrow for supporting user operations relating to thepanorama image imaging operation (swing operation of the imagingapparatus 100). That is to say, the user can perform the panorama imageimaging operation by swinging the imaging apparatus 100 in the directionwhich the arrow 334 indicates.

FIG. 10A shows a state in which the user 420 has moved the imagingapparatus 100 to the end position 422 of the imaging operation, in asimplified manner. FIG. 10B shows an operation assisting screen 335displayed on the input/output panel 200 at the time of the user 420having moved the imaging apparatus 100 to the end position 422 of theimaging operation. As shown in FIG. 10B, upon the imaging apparatus 100having been moved to the end position 422 of the imaging operation andthe user 420 having performed an end instruction operation of theimaging operation, a message to the effect that the imaging operationhas ended is displayed on a message display region 336.

Also, operation supporting images 337 through 339 which indicate thevertical direction imaging range and the right edge imaging range aredisplayed on the operation assisting screen 335, and the arrow shown inFIG. 9B is erased.

Thus, in the event that the user 420 has moved the imaging apparatus 100to the end position 422 of the imaging operation, the imaging operationof the panorama image ends, and a message to the effect that thepanorama image imaging operation has ended is displayed. Endingprocessing of the panorama image imaging operation is performed by thecontrol unit 160 determining whether or not the imaging apparatus 100has reached the end position 422, based on the attitude information fromthe attitude detecting unit 150.

Also, a panorama image is generated by the image combining unit 120using the multiple images generated by the imaging operation. After thepanorama image imaging operation has ended, the generated panorama imageis recorded in the recording medium 140 by the recording control unit130 as a still image file.

Note that with the first embodiment of the present technology, anexample has been described where the imaging range (image size) of thepanorama image is set beforehand, and ending processing of the panoramaimage imaging operation is performed using this image size. Notehowever, that an arrangement may be made wherein the image size ischanged during the panorama image imaging operation, based on useroperations. For example, the image size can be changed by moving theoperation supporting image 338 indicating the imaging range at the rightedge to the left or right during the panorama image imaging operation.This moving operation can be performed by an operation using the dial192, for example, or a touch operation (tracing operation) on theinput/output panel 200. Accordingly, the user can adjust the imagingrange of the panorama image while confirming the imaging range thereof.

Note that there is a possibility that the user may not properly performthe swing operation while the imaging apparatus 100 moves from the startposition 423 of the imaging operation to the end position 422 of theimaging operation. For example, it can be expected that the imagingapparatus 100 will move somewhat in the vertical direction as well,during the swing operation in the horizontal direction. In this case,the display position of the operation supporting image is displayedmoved in accordance with the movement in the vertical direction. Also,in this case, a message may be displayed prompting a proper swingoperation, so as to notify the user to that effect.

Thus, by displaying operation supporting images indicating the imagingrange decided based on the deciding operation of the end position of thepanorama image imaging range, the user can easily comprehend the imagingrange of the panorama image before starting the panorama image imagingoperation. That is to say, the user can easily confirm the panoramaimage imaging range by performing the swing operation of the imagingapparatus 100.

Also, the operation supporting images are displayed until the panoramaimage imaging operation ends. Accordingly, the user can easily shoot anintended panorama image by swinging the imaging apparatus 100 followingthe operation supporting images. That is to say, a panorama imageaccording to user preferences can be easily generated.

Operation Example of Imaging Apparatus

Next, operations of the imaging apparatus 100 according to the firstembodiment of the present technology will be described with reference tothe drawings. FIGS. 11 and 12 are a flowchart illustrating an example ofprocessing procedures for imaging control processing by the imagingapparatus 100 according to the first embodiment of the presenttechnology.

First, the panorama image imaging mode is set (step S901). Next, thecontrol unit 160 determines regarding whether or not a decidingoperation for the end position for the panorama image imaging operationhas been performed (step S902), and in the event that no decidingoperation has been performed, monitoring is continued.

In the event that a deciding operation has been performed (Yes in stepS902), a positioned identified by the attitude of the imaging apparatus100 at the time of that deciding operation having been accepted is setas the end position (step S903). That is to say, the control unit 160decides the imaging range of the panorama image (horizontal directionand vertical direction), based on the end position. Note that step S902is an example of the accepting described in the Summary. Also, step S903is an example of the deciding described in the Summary.

Next, an imaged image is generated (step S904), and control unit 160calculates the amount of movement (movement angle) from the endposition, based on the attitude information from the attitude detectingunit 150 (step S905).

Next, the control unit 160 compares the amount of movement in thehorizontal direction and the amount of movement in the verticaldirection, with regard to the calculated amount of movement (step S906).In the event that the amount of movement in the horizontal direction isgreater than the amount of movement in the vertical direction (Yes instep S906), the display control unit 170 displays an operationsupporting image of which the horizontal direction is the direction ofmovement, superimposed on the LV image (step S908). For example, aportion of a rectangular dotted line of which the horizontal directionis the longitudinal direction (see FIGS. 7B and 8B, for example) isdisplayed on the display unit 180 as an operation supporting image.

On the other hand, in the event that the amount of movement in thehorizontal direction is not greater than the amount of movement in thevertical direction (No in step S906), the display control unit 170displays an operation supporting image of which the vertical directionis the direction of movement, superimposed on the LV image (step S907).For example, a portion of a rectangular dotted line of which thevertical direction is the longitudinal direction is displayed on thedisplay unit 180 as an operation supporting image.

Next, the control unit 160 determines whether or not a start instructionoperation (start position deciding operation) for the panorama imageimaging operation has been performed (step S909), and in the event thatthe start instruction operation has not been performed, the flow returnsto step 5904. In the event that the start instruction operation for thepanorama image imaging operation has been performed (Yes in step S909),the control unit 160 compares the amount of movement in the horizontaldirection and the amount of movement in the vertical direction (stepS910). In the event that the results of comparison show that the amountof movement in the horizontal direction is greater than the amount ofmovement in the vertical direction (Yes in step S910), the control unit160 makes settings with the horizontal direction as the direction ofmovement (swing direction) (step S912).

On the other hand, in the event that the amount of movement in thehorizontal direction is not greater than the amount of movement in thevertical direction (Yes in step S910), the control unit 160 makessettings with the vertical direction as the direction of movement (swingdirection) (step S911). An operation supporting image is displayed basedon the direction of movement (horizontal direction or verticaldirection) set in this way. For example, in the event that thehorizontal direction is set to be the direction of movement (swingdirection) (step S912), a portion of a rectangular dotted line of whichthe horizontal direction is the longitudinal direction (see FIGS. 8Bthrough 10B, for example) is displayed on the display unit 180 as anoperation supporting image.

Next, an imaged image is generated (step S913). Next, the displaycontrol unit 170 displays the operation supporting image on the displayunit 180 superimposed on the LV image, based on the direction ofmovement (horizontal direction or vertical direction) set by the controlunit 160 (step S914). Next, recording processing of the generated imagedimage is performed (step S915). With this processing, an imaged imageused for the panorama image (one imaged image subjected to imageprocessing by the image processing unit 113) is temporarily recorded.

Next, the control unit 160 determines whether or not an end instructionoperation of the panorama image imaging operation has been performed(step S916), and in the event that an end instruction operation has notbeen performed, the flow returns to step 5913. In the event that an endinstruction operation has been performed (Yes in step S916), the imagecombining unit 120 generates a panorama image using images subjected toimage processing by the image processing unit 113 (multiple imagestemporarily recorded) under control of the control unit 160 (step S917).Next, under control of the control unit 160, the recording control unit130 records the panorama image generated by the image combining unit 120in the recording medium 140 as an image file (step S918).

2. Second Embodiment

With the first embodiment according to the present technology, anexample has been illustrated in which the end position for the panoramaimage imaging operation is decided based on user operations. Since theend position for the panorama image imaging operation is decided basedon user operations, it can also be conceived to decide the end timing ofthe panorama image imaging operation using the imaged image at the timeof that user operation (end position). Accordingly, with the secondembodiment according to the present technology, an example will beillustrated where the end timing of the panorama image imaging operationis decided using the imaged image at the end position of the panoramaimage imaging operation.

Configuration Example of Imaging Apparatus

FIG. 13 is a block diagram illustrating a functional configurationexample of an imaging apparatus 500 according to the second embodimentof the present technology. Note that the imaging apparatus 500 is amodification of the imaging apparatus 100 shown in FIG. 1. Accordingly,parts which are in common with those of the imaging apparatus 100 willshare the same nomenclature and description of a part thereof will beomitted.

The imaging apparatus 500 has a matching image detecting unit 510 and acontrol unit 520. The matching image detecting unit 510 is forperforming matching processing regarding images generated by the imagingdevice 112, and outputs the detection results thereof to the controlunit 520. Specifically, in the event that a deciding operation for theend position of the panorama image imaging operation is performed, thematching image detecting unit 510 holds the image generated by theimaging device 112 (the image subjected to image processing by the imageprocessing unit 113). The matching image detecting unit 510 thendetermines whether or not the image that is held (held image) and animage generated by the imaging device 112 following the attitude of theimaging apparatus 500 changing after accepting that deciding operation,and detects matching images.

The control unit 520 performs control to stop the panorama image imagingoperation based on the detection results from the matching imagedetecting unit 510. Specifically, the control unit 520 determineswhether the aforementioned held image and the image generated by theimaging device 112 during the panorama image imaging operation match,based on the detection results from the matching image detecting unit510. In the event that these images match, the control unit 520 performscontrol to stop the panorama image imaging operation.

Example of Holding Image

FIGS. 14A and 14B are diagrams schematically illustrating an example ofimage holding in a case of holding an image in the matching imagedetecting unit 510 according to the second embodiment of the presenttechnology. FIG. 14A shows an operation assisting screen 315 displayedin a case of a deciding operation being performed to decide the endposition of a panorama image imaging operation. Note that the operationassisting screen 315 shown in FIG. 14A is the same as that in FIG. 6B,so description here will be omitted.

FIG. 14B illustrates an image (held image 511) held in the matchingimage detecting unit 510, in a simplified manner. As shown in FIGS. 14Aand 14B, the image (held image 511) generated by the imaging device 112at the time of the deciding operation of the end position of thepanorama image imaging operation and subjected to image processing bythe image processing unit 113, is held in the matching image detectingunit 510. The matching image detecting unit 510 performs matchingprocessing using the held image 511 with regard to the imaged imagegenerated during the panorama image imaging operation. This matchingprocessing will be described in detail with reference to FIGS. 15A and15B.

Example of Detecting Matching Image

FIGS. 15A and 15B are diagrams schematically illustrating matchingprocessing with the matching image detecting unit 510 according to thesecond embodiment of the present technology. FIG. 15A schematicallyillustrates matching processing for detecting an image matching the heldimage 511 (a matching image) during the panorama image imagingoperation. Specifically, this illustrates the relation between animaging range (panorama image imaging range 530) which is the object ofimaging for the panorama image imaging operation, and an imaging range531 which is the object of imaging with the imaging device 112. Also,matching processing 532 to determine whether or not the imaging range531 and the held image 511 match is schematically shown.

As shown in FIG. 15A, matching processing between the imaged imagesgenerated while the swing operation of the imaging apparatus 500 isbeing performed (images shown within imaging range 531) and the heldimage 511 is sequentially performed.

Now, matching processing is processing in which the feature amounts oftwo images to be compared are compared, similarity of the featureamounts of the two images is calculated based on the comparison results,and determination is made regarding whether the two images match or notbased on this similarity. For example, in the event that the calculatedsimilarity is below a threshold, the two images to be compared aredetermined to not match, and in the event that the calculated similarityis at or above the threshold, the two images to be compared aredetermined to match.

FIG. 15B schematically illustrates a case where an image matching theheld image 511 (an image shown within the imaging range 535) has beendetected in the matching processing 536. As shown in FIG. 15B, in theevent that an image matching the held image 511 (image shown within theimaging range 535) has been detected in the matching processing 536, thematching image detecting unit 510 outputs a notification to that effectto the control unit 520. Upon receiving the notification, the controlunit 520 performs end processing to end the panorama image imagingoperation. That is to say, the imaging operation ends after imagingoperation of an image corresponding to the end position of the panoramaimage imaging operation has ended.

Now, in the event that an image matching the held image 511 is notdetected in the matching processing, end processing is performed at thepanorama image imaging range set beforehand (the end position (or aposition advanced a predetermined angle from the end position)).

Also, while an example has been illustrated in which matching processingis sequentially performed while the swing operation of the imagingapparatus 500 is being performed, an arrangement may be made whereinmatching processing is only performed within a certain range from theend position of the panorama image imaging operation that has been setbeforehand. For example, matching processing may be started in the eventof reaching within a range of ±5° from the end position. Accordingly,the load on the imaging apparatus 500 when performing imaging operationscan be reduced. Also, matching processing can be suitably performed evenin a case where a great part of the panorama has few features.

Operation Example of Imaging Apparatus

FIGS. 16 and 17 are a flowchart illustrating an example of processingprocedures for imaging control processing by the imaging apparatus 500according to the second embodiment of the present technology. FIGS. 16and 17 are a modification of FIGS. 11 and 12, so portions common toFIGS. 11 and 12 will be denoted with the same reference numerals, andpart of the description thereof will be omitted.

After the end position of the panorama image imaging operation has beenset (step S903), the matching image detecting unit 510 holds an imagegenerated by the imaging device 112 at the time of the decidingoperation having been accepted (step S921).

Also, after recording processing of the generated imaged image has beenperformed (step S915), the matching image detecting unit 510 determineswhether or not the generated image and the image that is held (heldimage) match (step S922). In the event that the generated image and theheld image do not match (No in step S922), the flow returns to stepS913. On the other hand, in the event that the generated image and theheld image match (Yes in step S922), the matching image detecting unit510 outputs a notification to that effect to the control unit 160. Thus,end processing of the panorama image imaging operation is performed, andthe image combining unit 120 generates a panorama image (step S917).

3. Third Embodiment

With the first embodiment according to the present technology, anexample has been illustrated in which the deciding operation of the endposition and start position for the panorama image imaging operation isperformed manually. Now, in the event that the image size (angle) of thepanorama image has been set beforehand, it can be conceived toautomatically decide the start position and end position of the panoramaimage imaging operation by deciding a predetermined position on thepanorama image (e.g., intermediate position). Accordingly, with thethird embodiment of the present technology, an example will beillustrated of automatically deciding the start position and endposition of the panorama image imaging operation by deciding apredetermined position on the panorama image. Note that theconfiguration of the imaging apparatus according to the third embodimentaccording to the present technology is generally the same as that of theexample shown in FIG. 1. Accordingly, portions which are in common withthe first embodiment according to the present technology are denotedwith the same reference numerals, and description thereof will beomitted.

Example of Panorama Image Imaging Operation

FIGS. 18A and 18B are images illustrating an example of a relationbetween the attitude of the imaging apparatus 100 according to the thirdembodiment of the present technology, and the display screen displayedon the input/output panel 200. Note that in the same way as with FIG. 5Aand other drawings, only the buildings 401 and 402 and the large tree403 in the place 400 shown in FIG. 4A are shown, and others are omittedfrom illustration, to facilitate description, in FIG. 18A. Also, FIGS.18A and 18B illustrate an example of imaging operations in a case wherea 180° image size has been set beforehand.

FIG. 18A illustrates a state of the imaging apparatus 100, and the user420 in a state of holding the imaging apparatus 100 in both hands, asviewed from above in a simplified manner. We will say that the user 420has confirmed an intermediate position in the panorama image (a positionequivalent to the center of the panorama image) while watching thesubject displayed on the input/output panel 200, and has performed adeciding operation to decide the intermediate position of the panoramaimage. This deciding operation can be performed by a full-pressingoperation of the shutter button 191.

For example, the user 420 decides the intermediate position in thepanorama image while viewing the LV image displayed on the input/outputpanel 200. In the event that an imaging range including the building 401is to be decided as the intermediate position as shown in FIG. 18A, forexample, the user 420 performs a full-press operation (decidingoperation) of the shutter button 191 in the state shown in FIG. 18A. Inthe event that an intermediate position deciding operation has beenperformed in this way, the control unit 160 takes the attitude of theimaging apparatus 100 at the time of that deciding operation having beenmade as a reference attitude. That is to say, the attitude relating tothe attitude information output from the attitude detecting unit 150 atthe time of the deciding operation having been made is decided as beingthe reference attitude of the imaging apparatus 100. Note that anexample of a display screen displayed on the input/output panel 200after the user 420 having decided the intermediate position of thepanorama image imaging operation is shown in FIG. 18B.

Now, description will be made regarding a method for deciding the startposition and end position of the imaging operation in the horizontaldirection, with the set image size being θ1 (180°). Now, we will saythat the user 420 has performed a full-press operation of the shutterbutton 191 in the state shown in FIG. 18A. In this case, a startposition 603 and end position 604 of the imaging operation is decidedbased on the set image size θ1, with the optical axis direction from aposition 601 at the time of making the operation (direction ofintermediate position 602) as a reference. Specifically, based on theimage size θ1, an angle) θ2 (90°) from the intermediate position 602 tothe start position 603 of the imaging operation, and an angle θ3 (90°)from the intermediate position 602 to the end position 604 of theimaging operation, are decided. That is to say, θ2=θ1/2 and θ3=θ1/2 arecalculated.

Note that a position other than the intermediate position may be decidedas being the reference position. For example, in the event that 120° isdecided as the image size, 40° (i.e., 40° from the start position) maybe used as a reference position. In this case, θ2=θ1×θ(1/3) andθ3=θ1×θ(2/3) hold. Note that this reference position may be changed byuser operations.

FIG. 18B shows an operation assisting screen 610 displayed at the timeof the deciding operation to decide the intermediate position of thepanorama image being performed. The operation assisting screen 610 hasthe subject currently being imaged as an LV image, and also a messagedisplay region 611 and operation supporting images 612 through 614 beingdisplayed superimposed on the LV image.

The message display region 611 displays a message to the effect that theintermediate position of the panorama image imaging operation has beendecided, and displays a message prompting the user to move the imagingapparatus 100 to the start position of the panorama image imagingoperation.

The operation supporting images 612 and 613 are dotted lines (overallrectangular dotted lines) indicating the panorama image imaging range(rectangular shape), in the same way as with the first embodiment of thepresent technology, and are disposed on the LV image displayed on theoperation assisting screen 610. Also, the operation supporting image 614is a solid line indicating the center position in the panorama image(center position in the swing direction), and is situated on the LVimage displayed on the operation assisting screen 610, in the same wayas with the first embodiment of the present technology.

Note that the method for displaying operation assisting screenscorresponding to change in the attitude of the imaging apparatus 100 isgenerally the same as with the first embodiment of the presenttechnology, so description thereof will be omitted here. Thus, apanorama image according to user preferences can be easily generatedeven in a case of deciding a position other than the end position of thepanorama image imaging operation (e.g., intermediate position) as areference position.

4. Fourth Embodiment

With the first embodiment according to the present technology, anexample of an imaging apparatus having a single imaging unit has beendescribed. Now, a case where a single imaging apparatus has multipleimaging units (e.g., a main camera and auxiliary camera) is conceivable.In this case, while the panorama image imaging operation can beperformed using one imaging unit (e.g., the main camera), using theother imaging unit (e.g., auxiliary camera) in the panorama imageimaging operation is conceivable. Accordingly, with the fourthembodiment according to the present technology, an example of performingthe panorama image imaging operation using an imaging apparatus havingmultiple imaging units will be illustrated.

Example of External Configuration of Imaging Apparatus

FIGS. 19A and 19B are external configuration diagrams showing externalviews of the imaging apparatus 700 according to the fourth embodiment ofthe present technology. FIG. 19A is a perspective view illustrating theexternal view of the front side of an imaging apparatus 700. Also, FIG.19B is an upper view illustrating the upper face of the imagingapparatus 700 (i.e., the face where the shutter button 191 and dial 192are provided). Note that portions other than a right side lens unit 701and a left side lens unit 702 are illustrated with dotted lines in asimplified manner, in FIG. 19B. Note that the imaging apparatus 700 is apartial modification of the imaging apparatus 100 shown in FIGS. 2A and2B. Accordingly, in FIGS. 19A and 19B, parts which are the same as withthe imaging apparatus 100 are denoted with the same reference numerals,and part of the description thereof will be omitted.

The imaging apparatus 700 has a right side lens unit 701 and a left sidelens unit 702. The right side lens unit 701 and left side lens unit 702are configured of multiple lenses which collect light from the subject(zoom lens, focusing lens, etc.), and are turnable in a particulardirection (the horizontal direction in the example shown in FIGS. 19Aand 19B). For example, the right side lens unit 701 may be turnable inthe horizontal direction by θ11 (degrees), and the left side lens unit702 turnable in the horizontal direction by θ12 (degrees). For example,θ11=θ12=90 (degrees) may be implemented.

In this case, a range from a position where the optical axis directionof the right side lens unit 701 and the optical axis direction of thelens unit 111 are parallel, to a position where the optical axisdirection of the right side lens unit 701 and the optical axis directionof the lens unit 111 are orthogonal, may be the turning range of theright side lens unit 701. In the same way, a range from a position wherethe optical axis direction of the left side lens unit 702 and theoptical axis direction of the lens unit 111 are parallel, to a positionwhere the optical axis direction of the left side lens unit 702 and theoptical axis direction of the lens unit 111 are orthogonal, may be theturning range of the left side lens unit 702.

Example of Functional Configuration of Imaging Apparatus

FIG. 20 is a block diagram illustrating a functional configurationexample of the imaging apparatus 700 according to the fourth embodimentof the present technology. Note that the imaging apparatus 700 is apartial modification of the imaging apparatus 100 shown in FIG. 1.Accordingly, in FIG. 20, parts which are the same as with the imagingapparatus 100 are denoted with the same reference numerals, and part ofthe description thereof will be omitted.

The imaging apparatus 700 includes the right side lens unit 701, theleft side lens unit 702, an imaging device 703, an imaging device 704,an image processing unit 711, an image combining unit 712, and a controlunit 713. The right side lens unit 701 and left side lens unit 702 arethe same as shown in FIGS. 19A and 19B. Light from the subject input viathe right side lens unit 701 and an iris are supplied to the imagingdevice 703, and light from the subject input via the left side lens unit702 and an iris are supplied to the imaging device 704.

The imaging device 703 converts the light input via the right side lensunit 701 (incident light from the subject) to generate an image (imagedata), and supplies the generated image to the image processing unit711. The imaging device 704 converts the light input via the left sidelens unit 702 (incident light from the subject) to generate an image(image data), and supplies the generated image to the image processingunit 711. Note that the imaging device 112 is an example of an imagingunit described in the Summary. Also, the imaging devices 703 and 704 areexamples of a second imaging unit described in the Summary.

That is to say, the lens unit 111 and imaging device 112 function as themain camera, and the right side lens unit 701 and imaging device 703,and the left side lens unit 702 and imaging device 704, function as anauxiliary camera (sub camera). Also, the auxiliary camera may besituated such that the optical axis direction of the main camera and theoptical axis direction of the auxiliary camera are different. Note thatcameras with different functions may be used as the main camera andauxiliary camera. For example, an arrangement may be made wherein themain camera is a camera having resolution of around 12 million pixels,while the auxiliary camera is a camera having resolution of around 2million pixels.

The image processing unit 711 processes various types of images withregard to the images (image data) generated by each of the imagingdevice 112, imaging device 703, and imaging device 704, under control ofthe control unit 713. Also, the image processing unit 711 suppliesimages (image data) subjected to image processing to the image combiningunit 712 and display control unit 170.

The image combining unit 712 combines a panorama image using multipleimage supplied from the image processing unit 711, and supplies thegenerated panorama image to the recording control unit 130, undercontrol of the control unit 713.

In the event that a deciding operation is accepted in the state of theoptical axis direction of the auxiliary camera facing the start positionor end position of the panorama image imaging operation, the controlunit 713 decides the panorama image imaging range based on thepositional relation between the main camera and auxiliary camera. Thedeciding operation is a deciding operation where the intermediateposition in the panorama image is decided to be the reference position.

Example of Screen for Setting Method for Shooting Panorama Image

FIG. 21 is a diagram illustrating a setting screen (shooting methodsetting screen 730) displayed on the input/output panel 200 in thefourth embodiment according to the present technology. The shootingmethod setting screen 730 is a screen displayed on the input/outputpanel 200 at the time of setting the panorama image shooting method. Forexample, the shooting method setting screen 730 is displayed immediatelyafter the panorama image imaging mode setting operation has beenperformed. The shooting method setting screen 730 includes selectionbuttons 731 through 734, an OK button 735, and a return button 736.

The selection buttons 731 through 734 are buttons used for setting thepanorama image shooting method. Now, the panorama image shooting methodcorresponding to the setting button 734 corresponds to the shootingmethod illustrated with the first embodiment of the preset technology.Also, the panorama image shooting methods corresponding to selectionbuttons 731 through 733 will be described in detail with reference toFIGS. 22A through 24B. Note that in the following description, thepanorama image shooting method corresponding to the selection button 731will be called “first shooting method”. Also, the panorama imageshooting method corresponding to the selection button 732 will be called“second shooting method”, and the panorama image shooting methodcorresponding to the selection button 733 will be called “third shootingmethod”.

The OK button 735 is a button pressed after a selection operation hasbeen made to select the panorama image shooting method, so as tofinalize that selection. Also, information relating to the panoramaimage shooting method decided by a pressing operation of the OK button735 (shooting method information) is output to the control unit 160 andheld. The return button 736 is a button pressed to return to the displayscreen displayed immediately before.

Example of Panorama Image Imaging Operation (First Shooting Method)

FIGS. 22A and 22B are diagrams schematically illustrating a transitionexample of the attitude of the imaging apparatus 700 according to thefourth embodiment of the present technology and a display screendisplayed on the input/output panel 200. Note that in the same way aswith FIG. 5A and other drawings, only the buildings 401 and 402 and thelarge tree 403 in the place 400 shown in FIG. 4A are shown, and othersare omitted from illustration, to facilitate description, in FIG. 22A.Also, FIGS. 22A and 22B illustrate an example of imaging operations in acase where a 180° image size has been set beforehand. Also, FIGS. 22Aand 22B illustrate an example of imaging operations in a case where afirst imaging method has been set (i.e., a case where a panorama imageshooting method corresponding to the selection button 731 in FIG. 21 hasbeen set).

Now, in the event that the first shooting method has been set, we willsay that the optical axis directions of each of the right side lens unit701 and left side lens unit 702 are determined based on the image sizethat has been set. For example, in the event that the image size thathas been set is θ1, the optical axis directions of each of the rightside lens unit 701 and left side lens unit 702 are each determined to bea position turned by θ1/2 from the optical axis direction of the lensunit 111. For example, in the event that the set image size is 180° asshown in FIG. 22A, the optical axis directions of each of the right sidelens unit 701 and left side lens unit 702 are determined to be positionsturned by 90° from the optical axis direction of the lens unit 111. Thatis to say, the optical axis direction of the lens unit 111 and theoptical axis directions of the right side lens unit 701 and left sidelens unit 702 are each determined to be at orthogonal positions. Notethat the right side lens unit 701 and left side lens unit 702 can beautomatically moved in accordance with the image size, by one ormultiple driving units (not shown), under control of the control unit713.

FIG. 22A illustrates a state of the imaging apparatus 700 in which theoptical axis direction of the lens unit 111 is facing the direction ofthe building 401, as viewed from above, in a simplified manner. FIG. 22Bshows an operation assisting screen 750 displayed in the event that adeciding operation to decide the intermediate position of the panoramaimage (deciding operations of start position and end position) has beenperformed.

The operation assisting screen 750 has a start position subject displayregion 751, an end position subject display region 752, a messagedisplay region 753, and operation supporting images 754 and 755,superimposed on the LV image. Note that the LV image is an imagegenerated by the imaging device 112 based on the subject light input viathe lens unit 111 (i.e., an image including the building 401). That isto say, this is an image including the subject which is currently beingimaged by the main camera.

The start position subject display region 751 has displayed therein animage generated by the imaging device 704 based on the subject lightinput via the left side lens unit 702 (i.e., an image including thebuilding 402). The end position subject display region 752 has displayedtherein an image generated by the imaging device 703 based on thesubject light input via the right side lens unit 701 (i.e., an imageincluding the large tree 403).

The message display region 753 has displayed therein a message to theeffect that the subjects existing at the start position and end positionof the panorama image imaging operation are displayed, and a messageprompting the user to move the imaging apparatus 700 to the startposition of the panorama image imaging operation. The operationsupporting images 754 and 755 are, in the same way as with the firstembodiment according to the present technology, images (dotted lines)indicating the imaging range of the panorama image in the verticaldirection.

Now, the user can confirm the position of the panorama image whilewatching the subjects displayed on the input/output panel 200 (thesubjects existing at the start position, intermediate position, and endposition), and perform a deciding operation to decide the intermediateposition (start position and end position) of the panorama image. Thisdeciding operation can be performed by full-pressing of the shutterbutton 191, in the same way as with the first embodiment according tothe present technology.

For example, the user can decide the imaging range corresponding to thepanorama image while viewing the LV image displayed on the input/outputpanel 200. For example, as shown in FIG. 22A, in a case where theposition of the building 402 is to be the start position 742 and theposition of the large tree is to be the end position 743, the userperforms a full-press operation of the shutter button 191 (decidingoperation) in the state shown in FIG. 22A. In the event that such adeciding operation has been performed, the control unit 713 takes theattitude of the imaging apparatus 700 at the time that the decidingoperation was performed as being the reference attitude. That is to say,the attitude relating to the attitude information output from theattitude detecting unit 150 at the time of that deciding operation beingperformed is determined as the reference attitude of the imagingapparatus 700.

Now, the start position subject display region 751 and end positionsubject display region 752 may continue to display only the images fromthe time of the deciding operation, until the panorama image imagingoperation ends. Accordingly, the user performing the panorama imageimaging operation can easily visually comprehend the start position andend position of the panorama image imaging operation.

Thus, by displaying a subject equivalent to the start position 742 inthe start position subject display region 751, and displaying a subjectequivalent to the end position 743 in the end position subject displayregion 752, the user can easily comprehend the positions in the panoramaimage. Now, the method for displaying the operation supporting images inaccordance with the change in attitude of the imaging apparatus 700 isgenerally the same as with the first embodiment according to the presenttechnology, so description thereof will be omitted here.

Example of Panorama Image Imaging Operation (Second Shooting Method)

FIGS. 23A and 23B are diagrams schematically illustrating a transitionexample of the attitude of the imaging apparatus 700 according to thefourth embodiment of the present technology and a display screendisplayed on the input/output panel 200. Note that in the same way aswith FIG. 5A and other drawings, only the buildings 401 and 402 and thelarge tree 403 in the place 400 shown in FIG. 4A are shown, and othersare omitted from illustration, to facilitate description, in FIG. 23A.Also, FIGS. 23A and 23B illustrate an example of imaging operations in acase where a 180° image size has been set beforehand. Also, FIGS. 23Aand 23B illustrate an example of imaging operations in a case where asecond imaging method has been set (i.e., a case where a panorama imageshooting method corresponding to the selection button 732 in FIG. 21 hasbeen set).

Now, in the event that the second shooting method has been set, we willsay that the optical axis direction of the left side lens unit 702 isdetermined based on the image size that has been set. For example, inthe event that the image size that has been set is θ1, the optical axisdirections of the left side lens unit 702 is determined to be a positionturned by θ1/2 from the optical axis direction of the lens unit 111. Forexample, in the event that the set image size is 180° as shown in FIG.23A, the optical axis direction of the left side lens unit 702 isdetermined to be a position turned by 90° from the optical axisdirection of the lens unit 111. That is to say, the optical axisdirection of the lens unit 111 and the optical axis direction of theleft side lens unit 702 are determined to be at orthogonal positions.Note that the left side lens unit 702 can be automatically moved inaccordance with the image size, by a driving unit (not shown), undercontrol of the control unit 713.

FIG. 23A illustrates a state of the imaging apparatus 700 in which theoptical axis direction of the lens unit 111 is facing the direction ofthe building 401, as viewed from above, in a simplified manner. FIG. 23Bshows an operation assisting screen 760 displayed in the event that adeciding operation to decide the intermediate position (start position)of the panorama image has been performed.

The operation assisting screen 760 has a start position subject displayregion 761, a message display region 762, and operation supportingimages 763 and 764, superimposed on the LV image. Note that theoperation assisting screen 760 is generally the same as the operationsupporting image 750 illustrated in FIG. 22B, other than the point thatthe end position subject display region 752 illustrated in FIG. 22B isnot displayed therein. Accordingly, detailed description here will beomitted. Thus, by displaying a subject equivalent to the start position742 in the start position subject display region 761, the user caneasily comprehend the positions in the panorama image.

Example of Panorama Image Imaging Operation (Third Shooting Method)

FIGS. 24A and 24B are diagrams schematically illustrating a transitionexample of the attitude of the imaging apparatus 700 according to thefourth embodiment of the present technology and a display screendisplayed on the input/output panel 200. Note that in the same way aswith FIG. 5A and other drawings, only the buildings 401 and 402 and thelarge tree 403 in the place 400 shown in FIG. 4A are shown, and othersare omitted from illustration, to facilitate description, in FIG. 24A.Also, FIGS. 24A and 24B illustrate an example of imaging operations in acase where a 180° image size has been set beforehand. Also, FIGS. 24Aand 24B illustrate an example of imaging operations in a case where athird imaging method has been set (i.e., a case where a panorama imageshooting method corresponding to the selection button 733 in FIG. 21 hasbeen set).

Now, in the event that the third shooting method has been set, we willsay that the optical axis direction of right side lens unit 701 aredetermined based on the image size that has been set. For example, inthe event that the image size that has been set is 01, the optical axisdirections of the right side lens unit 701 is determined to be aposition turned by θ1/2 from the optical axis direction of the lens unit111. For example, in the event that the set image size is 180° as shownin FIG. 24A, the optical axis direction of the right side lens unit 701is determined to be a position turned by 90° from the optical axisdirection of the lens unit 111. That is to say, the optical axisdirection of the lens unit 111 and the optical axis direction of theright side lens unit 701 are determined to be at orthogonal positions.Note that the right side lens unit 701 can be automatically moved inaccordance with the image size, by a driving unit (not shown), undercontrol of the control unit 713.

FIG. 24A illustrates a state of the imaging apparatus 700 in which theoptical axis direction of the lens unit 111 is facing the direction ofthe building 401, as viewed from above, in a simplified manner. FIG. 24Bshows an operation assisting screen 770 displayed in the event that adeciding operation to decide the intermediate position (end position) ofthe panorama image has been performed.

The operation assisting screen 770 has a start position subject displayregion 771, a message display region 772, and operation supportingimages 773 and 774, superimposed on the LV image. Note that theoperation assisting screen 770 is generally the same as the operationsupporting image 750 illustrated in FIG. 22B, other than the point thatthe start position subject display region 751 illustrated in FIG. 22B isnot displayed therein. Accordingly, detailed description here will beomitted. Thus, by displaying a subject equivalent to the end position743 in the end position subject display region 771, the user can easilycomprehend the positions in the panorama image.

Note that an arrangement may be made where, using an image generatedwith the auxiliary camera, start processing of the imaging operation atthe start position of the panorama image imaging operation, or endprocessing of the imaging operation at the end position of the panoramaimage imaging operation, is automatically performed. For example, thesecond embodiment according to the present technology may be applied.Specifically, an image displayed in the start position subject displayregion (i.e., an image generated by the imaging device 704) is held, andthe start position of the panorama image imaging operation is detectedby matching processing between the held image and the images generatedby the imaging device 112. In the event that the start position has beendetected, the start processing of the panorama image imaging operationis automatically performed. In the same way, an image displayed in theend position subject display region (i.e., an image generated by theimaging device 703) is held, and the end position of the panorama imageimaging operation is detected by matching processing between the heldimage and the images generated by the imaging device 112. In the eventthat the end position has been detected, the end processing of thepanorama image imaging operation is automatically performed.

Example of Generating Panorama Image Using Image of Auxiliary Camera

So far, examples of using images generated by the main camera forgenerating a panorama image have been illustrated. Here, an example willbe described of using images generated by the main camera and imagesgenerated by the auxiliary camera to generate a panorama image.

FIGS. 25A through 25C are diagrams illustrating an example of combininga panorama image using images generated by multiple imaging unitsaccording to the fourth embodiment of the present technology. Herein, ina case of taking a scene with great differences in light and dark (e.g.,a sunset scene shown in FIG. 32A) as the panorama image imaging range,it can be assumed that the dynamic range of the imaging device 112 whichthe imaging apparatus 700 has will be insufficient. In a case ofinsufficient dynamical range of the imaging device 112 in this way,there may be cases where the bright portions in the panorama imagebecome saturated and exhibit whiteout, while the dark portions in thepanorama image become exhibit blackout.

Now, there exists a technique in which a combined image with an extendeddynamic range is generated, by combining a long-exposure-time image anda short-exposure-time image with different exposure times, for example.Note that the long-exposure-time image and short-exposure-time image areimaged images continuously generated under different exposureconditions. Specifically, the short-exposure-time image is an imagedimage of which the exposure time is relatively short, for example, withpriority on relatively bright portions of the subjects in the sameshooting scene. On the other hand, the long-exposure-time image is animaged image of which the exposure time is relatively long, for example,with priority on relatively dark portions of the subjects in the sameshooting scene. Accordingly, a combined image in which highlight andshadow gradients have been reproduced can be generated by combining thelong-exposure-time image and short-exposure-time image, and the dynamicrange can be expanded.

Accordingly, with the example shown in FIGS. 25A through 25C, an exampleis illustrated of expanding the dynamic range using images generated bymultiple imaging devices. FIG. 25A illustrates an example of generatinga panorama image using images generated by the imaging devices 112, 703,and 704.

We will say that, for example, an imaging range to be imaged by theimaging device 112 is a main camera imaging range 780. We will also saythat an imaging range to be imaged by the imaging device 704 is anauxiliary camera imaging range 781, and an imaging range to be imaged bythe imaging device 703 is an auxiliary camera imaging range 782.

For example, the imaging device 112 generates a normal imaging image,and the imaging devices 703 and 704 generate multiple imaged images withdifferent exposure conditions (long-exposure-time image andshort-exposure-time image). Specifically, the imaging devices 703 and704 generate a long-exposure-time image and a short-exposure-time imageby imaging the same subject twice in a row, with different exposureconditions from those of the imaging device 112 during the panoramaimage imaging operation.

The image combining unit 712 generates a panorama image with an expandeddynamic range in at least a partial region, by combining multiple imagesgenerated by the imaging devices 703 and 704 with the multiple imagesgenerated by the imaging device 112. For example, the image combiningunit 712 decides the combining ratio of pixels in accordance withluminance of the pixels making up the short-exposure-time image, andperforms high dynamic range combining processing regarding the imagesoutput from the image processing unit 711 (long-exposure-time image andshort-exposure-time image), based on this combining ratio. In this case,the image combining unit 712 performs positioning regarding the imagesoutput from the image processing unit 711 (normal imaged images,long-exposure-time images, and short-exposure-time images) by performingmatching processing, and thereby can perform combining processing.

FIG. 25B illustrates an example of generating a panorama image usingimages generated by the imaging devices 112 and 704. Also, FIG. 25Cillustrates an example of generating a panorama image using imagesgenerated by the imaging devices 112 and 703. Note that with theexamples in FIGS. 25B and 25C, an image can be generated wherein thedynamic range has been expanded regarding only a portion of the panoramaimage. In this case, either imaging operation can be selected inaccordance with the subject.

Example of Imaging Apparatus Having Auxiliary Cameras Capable of Turningin Direction Other than Horizontal Direction

So far, description has been made regarding an example of generatingpanorama images using auxiliary cameras turnable in the horizontaldirection. Now, in the event of generating panorama images of which thevertical direction is the longitudinal direction, generating panoramaimages using auxiliary cameras turnable in the vertical direction can beexpected. Accordingly, in this example, an example of an imagingapparatus having auxiliary cameras capable of turning in a directionother than the horizontal direction is illustrated.

FIGS. 26A and 26B are external configuration diagrams showing externalviews of the imaging apparatus 790 according to the fourth embodiment ofthe present technology. FIG. 26A is a perspective view illustrating theexternal view of the front side of an imaging apparatus 790. Also, FIG.26B is an upper view illustrating the upper face of the imagingapparatus 790 (i.e., the face where the shutter button 191 and dial 192are provided). Note that the imaging apparatus 790 is a partialmodification of the imaging apparatus 700 shown in FIGS. 19A and 19B.Accordingly, in FIGS. 26A and 26B, parts which are the same as with theimaging apparatus 700 are denoted with the same reference numerals, andpart of the description thereof will be omitted.

The imaging apparatus 790 has a right side lens unit 791 and a left sidelens unit 792. The right side lens unit 791 and left side lens unit 792are configured of multiple lenses which collect light from the subject(zoom lens, focusing lens, etc.), and are turnable in various directions(e.g., with the example in FIGS. 26A and 26B, the horizontal directionand vertical direction). For example, we will say that the right sidelens unit 791 is turnable in the direction of the arrow 793 in thehorizontal direction, and is turnable in the direction of the arrow 794in the vertical direction. Also, we will say that the left side lensunit 792 is turnable in the direction of the arrow 795 in the horizontaldirection, and is turnable in the direction of the arrow 796 in thevertical direction. Note that the right side lens unit 791 and left sidelens unit 792 can be automatically moved in accordance with the imagesize, by a driving unit (not shown), under control of the control unit713.

Thus, the fourth embodiment of the present technology can be applied toan imaging apparatus having auxiliary cameras turnable in directionsother than the horizontal direction.

5. Modification

With the first through fourth embodiments, examples have beenillustrated in which operation supporting images indicating a panoramaimage imaging range are displayed. This modification illustrates anotherdisplay example of operation supporting images. Note that thismodification is a partial modification of the imaging apparatus 100shown in FIG. 1. Accordingly, with this modification, parts which arethe same as with the imaging apparatus 100 are denoted with the samereference numerals, and part of the description thereof will be omitted.Also, with this modification, the message display region displayed inthe operation supporting images will be omitted from illustration.

Example of Changing Color Outside Panorama Image Imaging Range

FIGS. 27A and 27B are diagrams illustrating a display example of anoperation assisting screen displayed on the imaging apparatus 100according to a modification of an embodiment of the present technology.FIG. 27A illustrates a display example of an operation assistant screen800 displayed in the case of imaging a subject equivalent to the centerposition of the panorama image (center position in the swing direction).Note that the operation assistant screen 800 is generally the same asthe operation assistant screen 320 illustrated in FIG. 7B and theoperation assistant screen 610 illustrated in FIG. 18B, so descriptionthereof will be omitted here.

FIG. 27B shows a display example of an operation assistant screen 805displayed in a case where a subject equivalent to near the startposition in the panorama image is to be imaged. With the example in FIG.27B, a case is illustrated where the imaging apparatus 100 is swung pastthe start position in the panorama image. In this case, an operationsupporting image 807 indicating the imaging range at the left end of thepanorama image is displayed toward the middle of the screen.

Also, with the display example in FIG. 27B, the color outside of thepanorama image imaging range is changed. For example, a region 809outside the panorama image imaging range may be made to be gray. Thus,by displaying with a different form, the user can be notified to theeffect that the region 809 outside the panorama image imaging range is aregion which is likely not to be included in the panorama image. Exampleof Displaying Angle of Panorama Image Imaging Range

FIGS. 28A and 28B are diagrams illustrating a display example of anoperation assisting screen displayed on the imaging apparatus 100according to a modification of an embodiment of the present technology.FIGS. 28A and 28B illustrate an example of displaying anglescorresponding to the panorama image imaging range.

FIGS. 28A and 28B illustrate a display example of operation assistantscreens 810 and 815 where angles corresponding to the panorama imageimaging range are displayed. An operation assistant screen 810 differsfrom the operation assistant screen 800 in FIG. 27A in that anglescorresponding to the panorama image imaging range are displayed, but isgenerally the same as with the operation assistant screen 800 regardingother points. Also, an operation assistant screen 815 differs from theoperation assistant screen 805 in FIG. 27B in that angles correspondingto the panorama image imaging range are displayed, but is generally thesame as with the operation assistant screen 805 regarding other points.Accordingly, portions which are the same as with the operation assistantscreens 800 and 805 will be described denoted with the same referencenumerals.

For example, angles corresponding to the panorama image imaging rangeare displayed along operation supporting images 802 and 808 at the lowerportion. These angles are angles in the case of the center position ofthe panorama image being 0°. Thus, the user can easily comprehend anaccurate imaging range by angles corresponding to the panorama imageimaging range being displayed.

Example of Displaying Operation Supporting Image Showing Center ofPanorama Image Imaging Range

FIGS. 29A and 29B are diagrams illustrating a display example of anoperation assisting screen displayed on the imaging apparatus 100according to a modification of an embodiment of the present technology.FIGS. 29A and 29B show an example of displaying an operation supportingimage indicating the center of the panorama image imaging range (centerin the direction orthogonal to the swing direction).

FIGS. 29A and 29B illustrate a display example of operation assistingscreens 820 and 825 indicating the center of the panorama image imagingrange. An operation assistant screen 820 differs from the operationassistant screen 810 in FIG. 28A in that an operation supporting image821 is displayed, but is generally the same as with the operationassistant screen 810 regarding other points. Also, an operationassistant screen 825 differs from the operation assistant screen 815 inFIG. 28B in that an operation supporting image 826 is displayed, but isgenerally the same as with the operation assistant screen 815 regardingother points. Accordingly, portions which are the same as with theoperation assistant screens 810 and 815 will be described denoted withthe same reference numerals.

For example, in the event that the swing direction is the horizontaldirection, the operation supporting images 821 and 826 indicating thecenter of the panorama image imaging range in the vertical direction. Bythus displaying the operation supporting images 821 and 826, the usercan easily comprehend the swing direction of the panorama image and thecenter of the panorama image imaging range in the vertical direction.

Example of Displaying Operation Supporting Image IndicatingPredetermined Range from Reference Position in Panorama Image

FIGS. 30A and 30B are diagrams illustrating a display example of anoperation assisting screen displayed on the imaging apparatus 100according to a modification of an embodiment of the present technology.FIGS. 30A and 30B show an example of displaying an operation supportingimage indicating a predetermined range from the reference position inthe panorama image. This reference position may be, for example, thecenter position of the panorama image, or an edge position of thepanorama image.

FIGS. 30A and 30B illustrate a display example of operation assistingscreens 830 and 835 where an operation supporting image indicating thepredetermined range from the reference position in the panorama aredisplayed. An operation assistant screen 830 differs from the operationassistant screen 820 in FIG. 29A in that an operation supporting image831 is displayed, but is generally the same as with the operationassistant screen 820 regarding other points. Also, an operationassistant screen 835 differs from the operation assistant screen 825 inFIG. 29B in that an operation supporting image 836 is displayed, but isgenerally the same as with the operation assistant screen 825 regardingother points. Accordingly, portions which are the same as with theoperation assistant screens 820 and 825 will be described denoted withthe same reference numerals.

For example, as shown in FIG. 30A, the operation supporting image 831indicating a predetermined range (e.g.,) ±10°) from a center position(the center position in the swing direction) in the panorama image isdisplayed. Also, for example, as shown in FIG. 30B, the operationsupporting image 836 indicating a predetermined range (e.g.,)±10°) froma left edge position (the left edge position in the swing direction) inthe panorama image is displayed. Thus, by displaying the operationsupporting images 831 and 836, the user can easily comprehend that thisis a predetermined range from the reference position of the panoramaimage.

Display Example of Operation Supporting Image in Case that SwingDirection is Vertical Direction

So far, display examples of operation supporting images have beenillustrated with regard to cases where the swing direction is thehorizontal direction. However, as described above, embodiments of thepresent technology can be applied to cases where the swing direction isthe vertical direction, as well. Accordingly, FIG. 31 illustrates adisplay example of operation supporting images in a case where the swingdirection is the vertical direction. FIG. 31 is a diagram illustrating adisplay example of an operation assisting screen displayed on theimaging apparatus 100 according to a modification of an embodiment ofthe present technology.

Note that an operation assisting screen 840 shown in FIG. 31 is theoperation assisting screen 830 shown in FIG. 30A that has been changedto an operation assisting screen in a case that the swing direction isthe vertical direction, and is generally the same as the operationassisting screen 830 other than this point. That is to say, theoperation supporting images 841 through 845 correspond to the operationsupporting images 801 through 803 and 821 and 831 shown in FIG. 30A.

Thus, by displaying various types of operation supporting images,panorama image imaging operations can be performed easily, and apanorama image according to user preferences can be easily generated.Also, these operation supporting images may be changeable by useroperations. Accordingly, the user can perform panorama image imagingoperations using operation supporting images according to userpreferences.

Example of Performing Exposure Control Using Information of Brightnessfrom End Position to Start Position of Panorama Image Imaging Operation

FIGS. 32A and 32B are diagrams illustrating the relation between theimaging range of a panorama image generated by an imaging apparatus 860according to a modification of an embodiment of the present technology,and an imaging operation.

FIG. 32A shows a panorama image imaging range 850 in a case ofgenerating a panorama image by an imaging apparatus 860, with a sunsetscene as the subject. Generally, the imaging range of a panorama imageis a relatively wide range, so relatively bright portions and relativelydark portions may coexist within the panorama image. For example, asillustrated in FIG. 32A, in the case of taking a panorama image with asunset scene as the subject, a relatively bright portion (bright region851) and a relatively dark portion (dark region 852) often coexist inthe panorama image imaging range 850.

In such a case, the user 861 can be expected to find a portion in thepanorama image imaging range 850 with brightness that is just right, andto perform exposure adjustment by manual operations. However, it canalso be expected that finding a portion in the panorama image imagingrange 850 with brightness that is just right will be difficult.

FIG. 32B illustrates the panorama image imaging operation in a case ofgenerating a panorama image with an imaging apparatus 860. Now, with thefirst and second embodiments of the present technology, examples havebeen described wherein the imaging apparatus is moved from the endposition to the start position of the panorama image imaging operation,and then the imaging apparatus is moved from the start position to theend position. That is to say, the imaging apparatus 860 makes onereciprocal movement between the start position and end position, asindicated by swing directions 863 and 864 in FIG. 32B. In this case,with a swing operation from the end position to the start position(swing direction 863), the imaging apparatus 860 can gather varioustypes of information in the panorama image imaging range 850.Accordingly, with this modification, and example will be illustratedwherein brightness information of the panorama image imaging range 850is gathered during the swinging of the imaging apparatus 860 from theend position to the start position of the panorama image imagingoperation, and this obtained brightness information is used toautomatically set the exposure for the panorama image.

For example, a brightness according to user preferences is setbeforehand in a brightness setting screen. For example, in the event ofincluding sky in the panorama image imaging range, the panorama imagecan often be made to look relatively beautiful by locking the exposureat a somewhat brighter portion. Accordingly, in the event of includingsky in the panorama image imaging range, for example, settings can bemade in which the exposure is locked at a somewhat brighter portion. Inthe event of performing a panorama image imaging operation in a case ofexposure conditions having been set in this way, the imaging apparatus860 automatically sets exposure based on the exposure conditions.

Also, let us assume that the imaging apparatus 860 is an imagingapparatus callable of setting multiple imaging modes (e.g., scenerymode, sunset mode, night lights mode, snow mode, beach mode, underwatermode). In this case, exposure of the panorama image can be automaticallyset using brightness information obtained from the panorama imageimaging range 850, based on the imaging mode that has been set. Suchsettings allow a panorama image according to user preferences to beeasily generated.

Example of Panorama Image Imaging Operation in a Case of Moving on aStraight Line

With the first through fourth embodiments of the present technology,examples have been illustrated wherein the panorama image imagingoperation is performed by the user performing a swing operation of theimaging apparatus with the position of the imaging apparatus as areference. However, a case can be expected where a panorama imageimaging operation is performed by moving the imaging apparatus along astraight line generally parallel to the subject, for example.Accordingly, an example of performing a panorama image imaging operationby moving the imaging apparatus along a straight line generally parallelto the subject will be described below.

FIGS. 33A and 33B are diagrams schematically illustrating an imagingoperation performed at the time of generating a panorama image by animaging apparatus 870 according to a modification of an embodiment ofthe present technology. FIG. 33A schematically illustrates a state ofthe imaging operation to generate a panorama image by rotating theimaging apparatus 870 in the horizontal direction (swing direction 873)with the position of the imaging apparatus 870 (position of user 871) asthe center of rotation, as viewed from above. Also, in FIG. 33A, thepanorama image imaging range of the panorama image generated by thisimaging operation is schematically illustrated with a solid line aspanorama image imaging range 872. Note that the example shown in FIG.33A is the same as examples shown in the first through fourthembodiments, so description thereof will be omitted here.

FIG. 33B schematically illustrates an imaging operation to generate apanorama image by moving the imaging apparatus 870 along a straight line883 generally parallel to the subject, as viewed from above. The imagingoperation shown in FIG. 33B is, for example, a panorama image imagingoperation performed using a moving device such as a vehicle or the likewhich can move at a constant speed, with the optical axis direction ofthe imaging apparatus 870 being orthogonal to the direction of movement881 thereof. For example, a vehicle which the user 871 is riding whileholding the imaging apparatus 870 in the hands moves in the direction ofmovement 881 at a constant or changing speed. Also, in FIG. 33B, thepanorama image imaging range of the panorama image generated by thisimaging operation is schematically illustrated with a solid line aspanorama image imaging range 882.

In this way, with the example shown in FIG. 33B, the position of theimaging apparatus 870 is sequentially changed along the straight line883, so the distance traveled along the straight line is used as changein the attitude of the imaging apparatus 870, instead of rotationalangle. Also, positions on the straight line are used for the startposition and end position of the panorama image imaging operation.

Note that while examples of an imaging apparatus having an imaging unithave been described with the embodiments of the present technology, theembodiments of the present technology can be applied to an imagingapparatus of which the imaging unit is detachable.

Also, while examples of generating a panorama image by combining stillimages have been described with the embodiments of the presenttechnology, the embodiments of the present technology can be applied toa case of generating a panorama image using moving images generated bymoving image imaging operations. Also, the embodiments of the presenttechnology can be applied to a case of generating a multi-view panoramaimage as well.

Also, the embodiments of the present technology can be applied to imageprocessing devices such as a cellular phone with imaging functions, apersonal digital terminal with imaging function, or the like.

Note that the above-described examples are examples of carrying out thepresent technology, and that there is correspondence between specificelements disclosed in the embodiments and the features in the claims. Inthe same way, there is correspondence between the features in the claimsand specific elements disclosed in the embodiments bearing the samenames. Note however, that the present technology is not restricted tothe embodiments, and can be carried out by making various modificationsto the embodiments without departing from the essence thereof.

Also, the processing procedures described in the above embodiments mayalso be understood to be a method having these series of procedures, andalso may be understood to be a program causing a computer to executethese series of procedures and a recording medium to stored the program.Examples of this recording medium include CD (Compact Disc), MD(MiniDisc), DVD (Digital Versatile Disk), memory card, Blu-ray Disc(registered trademark), and so forth.

Also, the present technology may have the following configurations.

-   (1) An apparatus including:

a reference position receiving unit configured to receive intermediateor end panorama reference position information input by a user; and

a control unit configured to control an imaging device to begingenerating a plurality of images to be used to generate a panoramicimage based on the intermediate or end panorama reference positioninformation input by the user after the reference position receivingunit receives the intermediate or end panorama reference positioninformation.

-   (2) The apparatus according to (1), wherein the intermediate    panorama reference position information includes a center position    of the panorama image.-   (3) The apparatus according to (1) or (2), further comprising:

the imaging device configured to generate the plurality of images.

-   (4) The apparatus according to (1), wherein the end panorama    reference position information includes a reference image, and the    control unit controls the imaging device to end generating the    plurality of images when a current image matches the reference    image.-   (5) The apparatus according to (1) to (4), further comprising:

an attitude detecting unit configured to detect a current orientation ofthe apparatus.

-   (6) The apparatus according to (5), wherein the end panorama    reference position information includes an end panorama reference    position orientation, and the control unit ends generating the    plurality of images when the current orientation matches the end    panorama reference position orientation.-   (7) The apparatus according to (5), wherein the control unit    receives an image width from the user and determines a start point    orientation based on the image width and the end panorama reference    position information.-   (8) The apparatus according to (5), wherein the intermediate    panorama reference position information includes an intermediate    panorama reference position orientation, and the control unit    receives an image width from the user and determines a start point    orientation and an end point orientation based on the image width    and the intermediate panorama reference position orientation.-   (9) The apparatus according to (1) to (8), wherein the control unit    receives an image width from the user and determines a start point    and an end point based on the image width and the intermediate    panorama reference position information.-   (10) The apparatus according to (3), wherein the imaging device    includes a main imaging device and a sub imaging device.-   (11) The apparatus according to (10), wherein the sub imaging device    images an end panorama reference image, and the control device ends    generating the plurality of images when a current image matches the    end panorama reference position image.-   (12) The apparatus according to (10), wherein the sub imaging device    images a start panorama reference image, and the control device    starts generating the plurality of images when a current image    matches the start panorama reference position image.-   (13) The apparatus according to (12), wherein the sub imaging device    images an end panorama reference image, and the control device ends    generating the plurality of images when a current image matches the    end panorama reference position image.-   (14) The apparatus according to (10), wherein the main imaging    device images all of the plurality of images for the panoramic    image.-   (15) The apparatus according to (1) to (14), wherein the apparatus    is a mobile phone, the mobile phone transmits the plurality of    images to a server, and receives the panoramic image from the    server.-   (16) The apparatus according to (1) to (15), wherein the control    unit is configured to control the imaging device to begin generating    the plurality of images to be used to generate the panoramic image    based on panorama reference position information for a plurality of    reference points input by the user after the reference position    receiving unit receives the panorama reference position information    for the plurality of reference points.-   (17) The apparatus according to (16), wherein the control unit    receives the panorama reference position information for the    plurality of reference points including at least a start point and a    center point.-   (18) The apparatus according to (1) to (17), wherein the control    unit is configured to control the imaging device to begin generating    the plurality of images to be used to generate the panoramic image    based on an end panorama reference position image input by the user    after the reference position receiving unit receives the end    panorama reference position image, and to control the imaging device    to end generating the plurality of images when a current image    matches the end panorama reference position image.-   (19) A method including:

receiving intermediate or end panorama reference position informationinput by a user; and

-   -   controlling an imaging device to begin generating a plurality of        images to be used to generate a panoramic image based on the        intermediate or end panorama reference position information        input by the user after the reference position receiving unit        receives the intermediate or end panorama reference position        information.

-   (20) A non-transitory computer readable medium encoded with a    program that, when loaded on a processor, causes the processor to    perform a method including:

receiving intermediate or end panorama reference position informationinput by a user; and

controlling an imaging device to begin generating a plurality of imagesto be used to generate a panoramic image based on the intermediate orend panorama reference position information input by the user after thereference position receiving unit receives the intermediate or endpanorama reference position information.

What is claimed is:
 1. An apparatus comprising: a reference positionreceiving unit configured to receive intermediate or end panoramareference position information input by a user; and a control unitconfigured to control an imaging device to begin generating a pluralityof images to be used to generate a panoramic image based on theintermediate or end panorama reference position information input by theuser after the reference position receiving unit receives theintermediate or end panorama reference position information.